Device and method of managing user information based on image

ABSTRACT

A device and a method of managing user information are provided. The device includes a display and a controller. The controller controls the display to display an avatar-based image according to user schedule information, change the avatar-based image according to a user input indicating a request to change the avatar-based image, and update the user schedule information according to the changed avatar-based image. The method includes displaying, on a display of a device, an avatar-based image according to user schedule information, changing the avatar-based image according to a user input indicating a request to change the avatar-based image, and updating the user schedule information according to the changed avatar-based image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.14/929,841 filed on Nov. 2, 2015 which claims priority from KoreanPatent Application No. 10-2014-0150635, filed on Oct. 31, 2014, andKorean Patent Application No. 10-2015-0070566, filed on May 20, 2015, inthe Korean Intellectual Property Office, the disclosures of which areincorporated by reference herein in their entirety.

BACKGROUND 1. Field

The present disclosure relates to devices and methods of managing userinformation, and more particularly, to a device and a method of managinguser information based on an image indicating user information.

2. Description of the Related Art

As communication technologies are being developed and smart devices arebecoming more widely used, various types of user information may becollected. The user information is information related to a user, suchas user profile information and/or life log information of the user. Thelife log information of the user may include user schedule informationor user health information, but is not limited thereto.

The user information may be collected by using a user input, a sensingfunction of a device, a detecting function of a device, and/or a dataanalyzing function of a device, but a method of collecting the userinformation is not limited thereto.

Technologies for effectively managing the user information collected assuch are required.

Information disclosed in this Background section was already known tothe inventors before achieving the inventive concept or is technicalinformation acquired in the process of achieving the inventive concept.Therefore, it may contain information that does not form the prior artthat is already known to the public in this country.

SUMMARY

It is an aspect to provide devices and methods of effectively managinguser information by easily changing, adding and/or checking userinformation based on an image indicating the user information.

It is another aspect to provide devices and methods of effectivelymanaging user information by easily checking and using a providableservice based on an image indicating user information.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented exemplary embodiments.

According to an aspect of an exemplary embodiment, there is provided adevice for managing user information, the device comprising a displayconfigured to display an avatar-based image (i.e. an image based on agraphical representation of a user and/or their characteristics); and acontroller configured to control the display such that the displaydisplays the avatar-based image according to user schedule information(i.e. the avatar-based image includes pictures, text or other forms ofinformation on past, present or future event or activity informationassociated with the user), change the avatar-based image according to auser input indicating a request to change the avatar-based image, andupdate the user schedule information according to the changedavatar-based image.

The device may comprise a user input module configured to receive a userinput requesting user information of a certain period, wherein thecontroller controls the display to display a plurality of avatar-basedimages according to the user information of the certain (i.e.predetermined) period in response to the user input, and the pluralityof avatar-based images are determined for and correspond respectively toa plurality of different periods that comprise (i.e. form at least partof) the certain period.

The controller may control the display to display a plurality ofavatar-based images, the device may further comprise a user input moduleconfigured to receive a user input requesting user information of acertain period, the controller may control the display to display theplurality of avatar-based images according to the user information ofthe certain period in response to the user input, and the plurality ofavatar-based images may be determined per theme item (i.e. an itemassociated with an activity or event) included in the user informationof the certain period.

The device may further comprise a user input module configured toreceive a user input requesting to display changeable informationrelated to the avatar-based image, wherein, in response to the userinput, the controller controls the display to distinguishably display(i.e. visually differentiate between) a changeable region and anunchangeable region (i.e. regions that may or may not be changed inresponse to user input) in the avatar-based image.

The controller may check an offering period (i.e. a period of time forwhich an avatar-based image is to be displayed by the device) that isset for the avatar-based image, and control the display to not displaythe avatar-based image when the offering period has expired.

The controller may control the display to display the avatar-based imageaccording to health information of a user, the device may furthercomprise a user input module configured to receive a user inputindicating a medical information request, and the controller may controlthe display to display medical information related to the healthinformation in response to the user input, and upon receiving, throughthe user input module, a user input about medical information selectedbased on the displayed medical information, updates the user scheduleinformation according to the selected medical information.

The device may further comprise a user input module configured toreceive a user input indicating information to be added to theavatar-based image, wherein the controller controls the display todisplay an input window for receiving an input of the information to beadded, and the controller adds an image corresponding to the informationreceived through the input window to the avatar-based image, and updatesthe user schedule information according to the avatar-based imageincluding the added image.

The controller may control the display to display the avatar-based imageaccording to future hope information (i.e. an ambition, long-term plan,or longer-term schedule) of the user, the device may further include auser input module configured to receive a user input requestinginformation related to the future hope information, and the controllermay control the display to display the information related to the futurehope information according to the user input, and updates the userschedule information according to information selected from theinformation related to the future hope information.

The controller may control the display to display a plurality ofavatar-based images according to usage history information of thedevice, the device may further include a user input module configured toreceive a user input requesting usage history information per usageperiod of the device, and the controller may control the display todisplay the plurality of avatar-based images in response to the userinput.

The controller may control the display to display the avatar-based imageaccording to user information registered in social media, the device mayfurther comprise a user input module for receiving a user inputrequesting to change the avatar-based image, and the controller maychange the avatar-based image in response to the user input, and updatethe user information registered in the social media according to thechanged avatar-based image.

According to another aspect of an exemplary embodiment, there isprovided a method of managing user information, wherein the method isperformed by a device, the method comprising displaying, on a display ofthe device, an avatar-based image according to user scheduleinformation; changing the avatar-based image according to a user inputindicating a request to change the avatar-based image; and updating theuser schedule information according to the changed avatar-based image.

The displaying may comprise displaying a plurality of avatar-basedimages according to user information of a certain period, wherein theplurality of avatar-based images are determined for and correspondrespectively to a plurality of different periods that comprise thecertain period.

The displaying may comprise displaying a plurality of avatar-basedimages according to user information of a certain period, wherein theplurality of avatar-based images are determined per theme item includedin the user information of the certain period.

The method may further comprise, in response to a user input requestingto display changeable information, distinguishably displaying achangeable region and an unchangeable region in the avatar-based image.

The method may further comprise checking an offering period that is setfor the avatar-based image; and when the offering period has expired,controlling the display to not display the avatar-based image.

The method may further comprise displaying, on the display, theavatar-based image according to health information of a user; providingmedical information related to the health information according to auser input indicating a medical information request; and updating theuser schedule information according to medical information selectedbased on the provided medical information.

The method may further comprise displaying, on the display, an inputwindow for receiving information to be added to the avatar-based imageupon receiving a user input indicating the information to be added;adding an image corresponding to information received through the inputwindow to the avatar-based image; and updating the user scheduleinformation according to the avatar-based image including the addedimage.

The method may further comprise displaying the avatar-based imageaccording to future hope information of a user; providing informationrelated to the future hope information according to a user inputrequesting the information related to the future hope information; andupdating the user schedule information according to information selectedfrom the information related to the future hope information.

The method may further comprise displaying a plurality of avatar-basedimages according to usage history per usage period of the deviceaccording to a user input requesting the usage history information.

The method may further comprise displaying the avatar-based imageaccording to user information registered in social media; changing theavatar-based image according to a user input requesting to change theavatar-based image; and updating the user information registered in thesocial media according to the changed avatar-based image.

According to another aspect, a computer program comprising instructionsarranged, when executed, to implement a method and/or apparatus inaccordance with any one of the above-described aspects is provided. Afurther aspect provides machine-readable storage storing such a program.

According to another aspect of an exemplary embodiment, there isprovided a method of visually managing and displaying user information,the method comprising displaying, on a display of a device, anavatar-based image that is modified according to user information;monitoring the device for a user input requesting a change to theavatar-based image; receiving an input of the change to the avatar-basedimage; displaying the changed avatar-based image; and updating the userinformation according to the changed avatar-based image.

The user information may be user schedule information.

The avatar-based image that is displayed may comprise a plurality ofimages, each depicting a different type of user schedule information.

The displaying the avatar-based image may comprise displaying aplurality of avatar-based images, each avatar-based image being modifiedaccording to user schedule information corresponding to a different timeperiod.

The different time periods may comprise weekend and weekdays.

The different time periods may comprise different days of a workweek.

The user information may comprises user health information.

The user health information may be received from an external device.

The user information may comprises usage information of the device.

It will be appreciated that that, although described as separate aspectsof the present disclosure, the aspects set out above may be combined inany combination. For example, the user input module may be configured toreceive user input requesting the display of changeable information anduser information of a certain period. Likewise, the controller may beconfigured to display a plurality of avatar-based images according touser information of the certain period and distinguishably display achangeable and an unchangeable region of one or more avatar-basedimages. Consequently, the user input module and controller may bothreceive and respond to a range and/or combination of user inputs.

It will be appreciated from the following description that, in certainaspects, features concerning the graphic design of user interfaces arecombined with interaction steps or means to achieve a technical effect.

It will also be appreciated that certain aspects aim to achieve atechnical effect of lowering a burden (e.g. a cognitive, operative,operational, operating, or manipulative burden) of a user whenperforming certain computer or device interactions. For example, via thepresentation and use of avatar-based images, information may be visuallyprovided to a user of a device and may be changed by the user of thedevice is an efficient and intuitive manner.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of the exemplary embodiments,taken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates an example of a device displaying an avatar-basedimage according to trip schedule information of a user, according to anexemplary embodiment;

FIG. 2 is a flowchart of a method of managing user information, whereina device changes user information by using an avatar-based imageaccording to user schedule information, according to an exemplaryembodiment;

FIG. 3 illustrates an example of user modeling information about userschedule information generated by a device, according to an exemplaryembodiment;

FIG. 4 is a table for describing a relationship between a plurality ofpieces of object information and a plurality of object images, which arestored in a device, according to an exemplary embodiment;

FIGS. 5A through 5E are diagrams for describing an example of a devicechanging an avatar-based image, according to an exemplary embodiment;

FIGS. 6A through 6E are diagrams for describing another example of adevice changing an avatar-based image, according to an exemplaryembodiment;

FIG. 7 is a flowchart of a method of managing user information, whereina device adds user information by using an avatar-based image accordingto user schedule information, according to an exemplary embodiment;

FIGS. 8A through 8C are diagrams for describing another example of adevice changing an avatar-based image, according to an exemplaryembodiment;

FIG. 9 is a flowchart of a method of managing, by a device, userinformation based on a plurality of avatar-based images according touser schedule information of a certain period, according to an exemplaryembodiment;

FIG. 10 illustrates an example of a device displaying a plurality ofavatar-based images based on weekday user schedule information andweekend user schedule information, according to an exemplary embodiment;

FIG. 11 is a flowchart of a method of managing user information, whereina device uses a plurality of avatar-based images based on user scheduleinformation per theme item, according to an exemplary embodiment;

FIGS. 12A through 12C are diagrams for describing an example of changinga plurality of avatar-based images per theme item, according to anexemplary embodiment;

FIG. 13 is a flowchart of a method of managing user information, whereina device manages a life of an avatar-based image, according to anexemplary embodiment;

FIG. 14 is a flowchart of a method of managing user information, whereina device uses an avatar-based image according to health information of auser, according to an exemplary embodiment;

FIGS. 15A through 15E are diagrams for describing an example ofupdating, by a device, user information by using an avatar-based imageaccording to health information of a user, according to an exemplaryembodiment;

FIG. 16 is a flowchart of a method of managing, by a device, userinformation based on an avatar-based image according to future hopeinformation of a user, according to an exemplary embodiment;

FIGS. 17A through 17D are diagrams for describing processes of updating,by a device, user schedule information based on an avatar-based imageaccording to future hope information of a user, according to anexemplary embodiment;

FIG. 18 is a flowchart of a method of managing user information based ona plurality of avatar-based images according to usage historyinformation of a device, according to an exemplary embodiment;

FIGS. 19A and 19B are diagrams for describing an example of providing,by a device, a plurality of avatar-based images according to a usagehistory information request of the device, according to an exemplaryembodiment;

FIG. 20 is a flowchart of a method of managing user information, whereina device uses an avatar-based image according to the user informationregistered in social media, according to an exemplary embodiment;

FIG. 21 is a block diagram of a system for managing user information,which includes a device, according to an exemplary embodiment;

FIG. 22 is a flowchart of a method of managing, by a device, userinformation, according to an exemplary embodiment;

FIGS. 23A through 23F are diagrams for describing avatar-based imagesdisplayed by a device based on user information, according to exemplaryembodiments;

FIG. 24A illustrates an example of user modeling information regardinguser information, according to an exemplary embodiment;

FIG. 24B illustrates an example of display sizes of an object includedin an avatar-based image displayed on a device, according to anexemplary embodiment;

FIG. 24C is a diagram for describing a display size of an objectincluded in an avatar-based image, according to an exemplary embodiment;

FIG. 25 illustrates an example of mapping information between an objectimage and object identification (ID) information, according to anexemplary embodiment;

FIG. 26 is a diagram for describing processes of generating anavatar-based image by using user modeling information and mappinginformation, according to an exemplary embodiment;

FIGS. 27A through 27F illustrate examples of a device displaying achangeable object, according to exemplary embodiments;

FIG. 28 is a flowchart of a method of changing, by a device, anavatar-based image according to a user input, according to an exemplaryembodiment;

FIGS. 29A through 29F illustrate examples of changing an object based oncandidate objects provided by a device, according to exemplaryembodiments;

FIG. 30 is a flowchart of a method of providing, by a device, serviceinformation providable according to objects included in an avatar-basedimage, according to an exemplary embodiment;

FIGS. 31A through 31D illustrate examples of a screen for describing themethod of FIG. 30;

FIG. 32 illustrates an example of user modeling information updated by adevice, according to an exemplary embodiment;

FIG. 33 is a flowchart of a method of managing, by a device, userinformation by changing an avatar-based image, according to an exemplaryembodiment;

FIG. 34 is a flowchart of a method of managing, by a device, userinformation by changing an avatar-based image, according to anotherexemplary embodiment;

FIG. 35 is a flowchart of a method of managing user information, whereina device changes and/or adds information by using an avatar-based image,according to an exemplary embodiment;

FIG. 36 is a flowchart of a method of managing user information suchthat a device is able to provide a service based on an object selectedfrom an avatar-based image, according to an exemplary embodiment;

FIG. 37 is a flowchart of a method of managing user information, whereina device provides a question and answer (QA) service based on anavatar-based image being displayed, according to an exemplaryembodiment;

FIG. 38 illustrates an example of a device providing answer informationaccording to received question information, according to an exemplaryembodiment;

FIG. 39 is a flowchart of a method of managing user information based ona device and an external device, according to an exemplary embodiment;

FIG. 40 is a flowchart of a method of managing user information based ona device and an external device, according to another exemplaryembodiment;

FIG. 41 is a flowchart of a method of managing user information based ona device and a wearable device, according to an exemplary embodiment;

FIG. 42 is a flowchart of a method of managing user information based ona device and a wearable device, according to another exemplaryembodiment;

FIGS. 43 and 44 are functional block diagrams of a device according toexemplary embodiments; and

FIG. 45 is a detailed functional block diagram of a controller accordingto an exemplary embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to exemplary embodiments, examplesof which are illustrated in the accompanying drawings, wherein likereference numerals refer to like elements throughout. In this regard,the present exemplary embodiments may have different forms and shouldnot be construed as being limited to the descriptions set forth herein.Accordingly, the exemplary embodiments are merely described below, byreferring to the figures, to explain aspects. In the description of thepresent disclosure, certain detailed explanations of related art areomitted when it is deemed that such explanations may unnecessarilyobscure the essence of the inventive concept.

While such terms as “first”, “second”, etc., may be used to describevarious components, such components must not be limited to the aboveterms. The above terms are used only to distinguish one component fromanother.

All terms including descriptive or technical terms which are used hereinshould be construed as having meanings that are obvious to one ofordinary skill in the art. However, the terms may have differentmeanings according to an intention of one of ordinary skill in the art,precedent cases, or the appearance of new technologies. Also, some termsmay be arbitrarily selected by the applicant, and in this case, themeaning of the selected terms will be described in detail in thedetailed description of the present disclosure. Thus, the terms usedherein have to be defined based on the meaning of the terms togetherwith the description throughout the specification.

An expression used in the singular encompasses the expression in theplural, unless it has a clearly different meaning in the context. In thepresent specification, it is to be understood that terms such as“including” or “having”, etc., are intended to indicate the existence ofthe features, numbers, steps, actions, components, parts, orcombinations thereof disclosed in the specification, and are notintended to preclude the possibility that one or more other features,numbers, steps, actions, components, parts, or combinations thereof mayexist or may be added.

Throughout the description and claims of this specification, the words“comprise” and “contain” and variations of the words, for example“comprising” and “comprises”, means “including but not limited to”, andis not intended to (and does not) exclude other moieties, additives,components, integers or steps.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

Throughout the description and claims of this specification, thesingular encompasses the plural unless the context otherwise requires.In particular, where the indefinite article is used, the specificationis to be understood as contemplating plurality as well as singularity,unless the context requires otherwise.

It will be also be appreciated that, throughout the description andclaims of this specification, language in the general form of “X for Y”(where Y is some action, activity or step and X is some means forcarrying out that action, activity or step) encompasses means X adaptedor arranged specifically, but not exclusively, to do Y.

Features, integers, characteristics, compounds, chemical moieties orgroups described in conjunction with a particular aspect, embodiment orexample are to be understood to be applicable to any other aspect,embodiment or example described herein unless incompatible therewith.

According to an exemplary embodiment, user information may include userprofile information.

The user profile information may include information about a gender of auser. The user profile information may include information about a nameof a user. The user profile information may include information about anage of a user. The user profile information may include informationabout a nationality of a user.

Also, the user profile information may include information about aregion a user is currently living. The user profile information mayinclude information about a company name a user is working for. The userprofile information may include information about a location of acompany a user is working for. The user profile information may includeinformation about a hobby of a user.

Also, the user profile information may include information about a placea user was born. The user profile information may include informationabout an occupation of a user. The user profile information may includeinformation about a family of a user, such as information about whetherthe user has parents and a child. The user profile information mayinclude information about a pet of a user, such as information aboutwhether the user owns a pet and about a type of a pet. The user profileinformation may include information about an interest of a user, such asclassic guitars, a sport, soccer, economics, trips, games, and books.However, the user profile information according to an exemplaryembodiment is not limited thereto.

Also, the user profile information may be referred to as user profileinformation (hereinafter, also referred to as social media profileinformation) registered in social media, such as Facebook, Twitter, aBlog, YouTube, or LinkedIn.

The social media profile information may include a keyword, aphotograph, a place, a time, and/or an event (for example, a photoexhibition, an art exhibition, a baseball game, a soccer game, aconcert, or a play), which are detected by analyzing context,vocabulary, an image, and/or a tag posted by a user, but is not limitedthereto.

The user information according to an exemplary embodiment may includeuser information (for example, membership information, game levelinformation, product information, usage information, and/or questinformation in a game) registered in a website (for example, Naver,Yahoo, or Google).

The user information according to an exemplary embodiment may includeuser information registered in each application installed in a device.The application installed in a device may be, for example, a scheduleapplication, a game application, a service center application, amembership application, a customer center application, a marketapplication, a life log managing application, a map application, or afuture hope information managing application (for example, a bucket listmanaging application or a wish list managing application), but is notlimited thereto.

According to an exemplary embodiment, the user information may includeinformation obtained by a device based on data generated or input/outputaccording to interaction between a device and a user.

The information obtained by a device may include search historyinformation. The information obtained by a device may include mailhistory information. The information obtained by a device may includemessage history information, wherein a message may be a short message, amulti message, or a voice message. The information obtained by a devicemay include browsing history information.

Also, the information obtained by a device may include social mediausage history information (or social network service (SNS) historyinformation). The information obtained by a device may include phonecall history information. The information obtained by a device mayinclude album history information. The information obtained by a devicemay include network history information. The information obtained by adevice may include content history information. However, the informationobtained by a device is not limited thereto.

The search history information may include at least one piece ofinformation obtained while a search application installed in a device isexecuted, for example, a list of search words (or query languages), thenumber of times each search word was searched for, a searched date,and/or a number of search results.

The mail history information may include at least one piece ofinformation obtained while a mail application installed in a device isexecuted, for example, a list of transmitted and received mails, thenumber of transmitted and received mails, the number of deleted mails, alist of senders, the number of transmitted mails per sender, a list ofrecipients, the number of received mails per recipient, and/or dates oftransmitted and received mails.

The message history information may include at least one piece ofinformation obtained while a message application installed in a deviceis executed, such as a list of transmitted and received messages, thenumber of transmitted and received messages, the number of deletedmessages, a list of senders, the number of transmitted messages persender, a list of recipients, the number of received messages perrecipient, and/or dates of transmitted and received messages.

The browsing history information may include at least one piece ofinformation obtained while a browsing application installed in a deviceis executed, such as a list of uniform resource locators (URLs), thenumber of times each URL was accessed, or URL link information.

The social media usage history information may include at least onepiece of information obtained while a social media service applicationinstalled in a device is executed, such as a number of postings and/or anumber of comments. For example, the social media usage historyinformation may include history information (for example, a list ofkeywords, the number of times each keyword was generated, a list ofplaces, the number of times each place was visited, activities of a userbased on an SNS according to times, and/or the number of times eachevent was viewed) based on at least one of a keyword, a photograph, aplace, a time and/or an event (for example, a photo exhibition, an artexhibition, a baseball game, a soccer game, a concert, or a play), whichare detected by analyzing at least one of context, vocabulary, an image,and/or a tag posted by a user.

The phone call history information may include at least one piece ofinformation obtained while a phone call application installed in adevice is executed, such as the number of times a video call was made,duration of a video call, a target of a video call, the number of timesa video call was made to each target, duration of a video call pertarget, the number of times a voice call was made, duration of a voicecall, a target of a voice call, the number of times a voice call wasmade to each target, and/or duration of a voice call per target.

The album history information may include at least one piece ofinformation about a still image and/or a moving image obtained while acamera application installed in a device is executed, such as a place atwhich the photograph or video was taken, photographed or videoed targetinformation (for example, a person, an object, or an animal), and/or adate/time at which the photograph or video was taken.

The network history information may include at least one piece ofnetwork connection information of a device. The network connectioninformation may include, for example, information on whether Wi-Finetwork is connected, the number of times Wi-Fi was accessed, a name ofan accessed Wi-Fi device, information about a Wi-Fi network, informationon whether Bluetooth network is connected, the number of times Bluetoothwas accessed, a name of an accessed Bluetooth device, data usage,information on whether near field communication (NFC) is contacted, thenumber of times NFC was contacted, details of using NFC, details ofaccessing a beacon, or/and details of accessing Zigbee, but the networkconnection information according to an exemplary embodiment is notlimited thereto.

The content history information may include content generated by using adevice and information about the content. The content historyinformation may include content shared with another device andinformation about the content. The content history information mayinclude content transmitted to another device and information about thecontent.

Also, the content history information may include content received fromanother device and information about the content. The content historyinformation according to an exemplary embodiment is not limited thereto.For example, information about content included in content historyinformation may include a date when content was generated, a data amountof content, a search keyword for content, a name of a device thatgenerated content, a name of a device that transmitted content, or aname of a device that received content.

According to an exemplary embodiment, the user information may includedevice configuration information set in a device. Device configurationinformation may include, for example, a name of a device, accountinformation, a backup account, and/or a name of an external device (or aperipheral device), but device configuration information according to anexemplary embodiment is not limited thereto.

According to an exemplary embodiment, the user information may be storedin a device. According to an exemplary embodiment, the user informationmay be received from at least one external device connected to a device.The at least one external device may include a wearable device, such asa smart watch or smart glasses. The at least one external device mayinclude an appcessory. The at least one external device may include amedical device. The at least one external device may include a vehicle.The at least one external device may include a server, such as aquestion and answer (QA) server or a game server. The at least oneexternal device may include an Internet of things (IoT) network-baseddevice (hereinafter, referred to as an IoT device). The at least oneexternal device may include an IoT network-based sensor (hereinafter,referred to as an IoT sensor). The at least one external device mayinclude a device, such as a game console.

The user information received from the at least one external device mayinclude a value sensed (or measured) by the at least one externaldevice. The user information received from the at least one externaldevice may include information collected by the at least one externaldevice. Information collected by the at least one external device mayinclude information detected or analyzed by the at least one externaldevice. However, the user information received from the at least oneexternal device is not limited thereto.

The value sensed by the at least one external device may include healthinformation of a user. The value sensed by the at least one externaldevice may include activity information of a user. However, the valuesensed by the at least one external device is not limited thereto.

The health information of a user may include information about exercisehours. The health information of a user may include information about aheart rate during exercise duration. The health information of a usermay include information about a pulse oximetry during exercise duration.The health information of a user may include information about calorieconsumption during exercise duration. The health information of a usermay include information about a moved distance of the user duringexercise duration.

Also, the health information of a user may include information about askin temperature of the user during exercise duration. The healthinformation of a user may include information about the number of stepstaken by the user in one period of time, such as in one day. The healthinformation of a user may include information about a galvanic skinresponse of the user during exercise duration. The health information ofa user may include information about duration of sleep of the user. Thehealth information of a user may include information about sleeppatterns of the user. The health information of a user may includeinformation about a blood-sugar level of the user.

Also, the health information of a user may include information sensed ormeasured by a medical device, such as information about bodycomposition, a vascular age, blood pressure, a meridian system, and/orpulses.

The activity information of a user may include information indicatingwalking. The activity information of a user may include informationindicating running. The activity information of a user may includeinformation indicating staying at one place. The activity information ofa user may include information indicating sleeping.

Also, the activity information of a user may include informationindicating a transport, such as a car, a bus, a train, an airplane, or aship. The activity information of a user may include information about atime the user stayed at one place. The activity information of a usermay include information about a place the user often visits.

Also, the activity information of a user may include information about aperson the user frequently sees. The activity information of a user mayinclude information about a meeting in which the user participated. Theactivity information of a user may include information about a gatheringin which the user participated. However, the activity information of auser according to an exemplary embodiment is not limited thereto. Theactivity information of a user may include a part of the healthinformation described above, such as exercise hours.

The information collected by the at least one external device mayinclude information collected by a server, but is not limited thereto.The information collected by the server may include information in whicha question of a user and a search result are mapped to each other. Theinformation collected by the server may include information obtained bythe device described above, such as a game level or an exercise level.The information collected by the server is not limited thereto.

According to an exemplary embodiment, the user information may includephysical information of a user. Also, the user information may includemind information of a user. The user information may include socialinformation of a user. The user information may include scheduleinformation of a user. The user information may include future hopeinformation of a user, such as bucket list or wish list information. Theuser information may include information about usage history informationper usage period of a device.

The physical information of a user may include information about a face,such as a face shape, an eye color, a nose shape, a mouth shape, an eyeshape, and/or an eyebrow shape. The physical information of a user mayinclude information about a height of the user. The physical informationof a user may include information about a hair style of the user, suchas short hair, long hair, curly hair, straight hair, or wavy hair. Thephysical information of a user may include face information of the user.The physical information of a user may include weight information of theuser. According to an exemplary embodiment, the physical information ofa user is not limited thereto. The physical information of a user mayinclude, for example, information about a gender and/or an age of theuser.

The mind information of a user (such as features of a user's personalityfor example) may include information about a taste of the user, such astaste in food. The mind information of a user may include informationabout tendency of the user, such as tendency information of the userbased on bookmark information, frequently executed applicationinformation, application execution time information, book purchaserecord information, video rental record information (or video on demand(VOD) download information), magazine subscription information, musicdownload information, and/or product purchase list information.

Also, the mind information of a user may include information about apreference of the user, such as preference information based on websitesearch list information and/or hobby information. The mind informationof a user may include information about feelings of the user, such asinformation indicating joy, sadness, fear, anger, surprise, greeting, ordisgust. The mind information of a user is not limited thereto. Forexample, the mind information of a user may include information aboutinterests described above.

The social information of a user may include information about anacademic background of the user. The social information of a user mayinclude information about a job of the user. The social information of auser may include information about a job description of the user. Thesocial information of a user may include information about a gatheringof the user. According to an exemplary embodiment, the socialinformation of a user is not limited thereto.

The schedule information of a user may include information input by theuser by using a schedule application but is not limited thereto. Theschedule information of a user may include information about the past,present, and future of the user. The schedule information of a user mayinclude user information based on at least one of time, day, month,year, and season.

The schedule information of a user may be classified based on themeitems. Examples of a theme item include at least one of a trip, acompany, study, a bike, a meeting, a date, a movie, an exercise,shopping, a hospital, beauty care, eating, a game, a gathering, hiking,a night life, a birthday party, a performance, or/and a business trip.According to an exemplary embodiment, the theme item is not limitedthereto.

The user information may include all or a part of user informationmanaged by personal information management software (PIMS). The userinformation may include at least one of all or a part of informationincluded in life log information of a user.

The user information may be expressed in a form of user modelinginformation or user personal information such that purposes (forexample, providing a user customized service) of a device, an externaldevice, or a service provider are effectively performed.

The user information may be expressed in an avatar-based image in a2-dimensional (2D) or 3D model. The user information may be expressed ina virtual image or a virtual self-graphic icon.

Hereinafter, one or more exemplary embodiments will be described indetail with reference to accompanying drawings.

FIG. 1 illustrates an example of a device 100 displaying an avatar-basedimage according to trip schedule information of a user and displayinginformation indicating a changeable image from the avatar-based image,according to an exemplary embodiment.

Referring to FIG. 1, a device 100 may display an avatar-based imageaccording to the trip schedule information of the user. The avatar-basedimage shown in FIG. 1 includes duration information 101 about dates of atrip, an avatar 102 indicating the user, a photo 103 of a trip, apicture 104 of a bag, information 105 indicating that the photo 103 is achangeable image, information 106 indicating that the picture 104 of thebag is a changeable image, and information 107 indicating that theduration information 101 is a changeable image. That is, as shown inFIG. 1, the information 105 through 107 indicating a changeable imagemay be a circle with a texture located therein at various positions onthe display screen of the device 100. However, this is only an example.

The avatar-based image according to the trip schedule information is notlimited to that shown in FIG. 1. For example, the avatar-based image mayinclude a drawing, a photo, or text regarding a plurality of trippreparations. The avatar-based image may omit the duration information101. The avatar-based image may further include information indicatingthat the avatar 102 is a changeable image.

The device 100 may indicate the information 105 through 107 of FIG. 1via another method. For example, the device 100 may change brightness ofa changeable image in a uniform or variable period. Alternatively, thedevice 100 may highlight a changeable image.

The device 100 may display information differently to indicate achangeable image according to images. For example, the device 100 maychange brightness of the photo 103 in a uniform or variable period,highlight the picture 104 of the bag, and periodically flicker theduration information 101.

FIG. 2 is a flowchart of a method of managing user information, whereinthe device 100 displays an avatar-based image according to user scheduleinformation and updates the user schedule information by changing theavatar-based image, according to an exemplary embodiment.

In operation S201, the device 100 displays an avatar-based imageaccording to user schedule information. The user schedule informationmay be information stored in a schedule application installed in thedevice 100, but is not limited thereto. For example, the user scheduleinformation may be received from an external device connected to thedevice 100.

The device 100 may generate the avatar-based image according toontology-based user modeling information. Ontology may describe arelationship between words, between a word and a concept, or betweenconcepts in a formal language.

The formal language may include, for example, a language such ashasHobby, hasSize, isA, or Cause, but the formal language according toan exemplary embodiment is not limited thereto. hasHobby is a formallanguage indicating that a relationship between words is a hobby.hasSize is a formal language indicating that a relationship betweenwords is a size. isA is a formal language indicating an inclusionrelation between words. Cause is a formal language indicating acause-and-effect relationship between words. For example, “need a restdue to overwork” may be expressed in (overwork, rest) by using Cause.

The user modeling information may be generated by an external deviceconnectable to the device 100.

FIG. 3 illustrates an example of user modeling information about userschedule information generated by the device 100, according to anexemplary embodiment.

The user modeling information of FIG. 3 is generated using resourcedescription framework (RDF) data modeling. The device 100 may generatethe user modeling information according to an ontology web language(OWL) or semantic web rule language (SWRL).

Referring to FIG. 3, the user modeling information includes a subjectfield, a predicate field, and an object field. The subject field is afield related to a subject of user information. In FIG. 3, a subject ofthe user information is a first user User_1. Here, it is shown that thefirst user User_1 is a person by using a class and a resource. Theresource defines an instance of the class.

The predicate field of FIG. 3 is a formal language field indicating arelationship between words included in the subject field and the objectfield. Referring to FIG. 3, the formal language included in thepredicate field includes a formal language about a gender, a formallanguage about a trip schedule, and a formal language about trippreparations. However, the formal language included in the predicatefield according to an exemplary embodiment is not limited thereto. Forexample, the formal language about a trip schedule may be referred to asa formal language about a schedule. The formal language about trippreparations may be referred to as a formal language about preparationsor trip check items.

The device 100 may express a female in the object field as a gender ofthe first user User_1 according to the formal language about a gender.The device 100 may express first through fifth trip places in the objectfield as trip places (or destinations) of the first user User_1according to the formal language about a trip schedule. The device 100may express first through fourth preparations in the object field astrip preparations of the first user User_1 according to the formallanguage about trip preparations.

The device 100 may generate an avatar-based image by using objectinformation (the female, the first through fifth trip places, and thefirst through fourth preparations) of the object field included in theuser modeling information. The object information of the object fieldmay be used as identification (ID) information for generating theavatar-based image.

In order to generate the avatar-based image, the device 100 may store aplurality of object images. The plurality of object images may be storedin an external device connectable to the device 100.

The plurality of object images may include the number of imagescorresponding to the number of pieces of the object information includedin the object field of FIG. 3. The plurality of object images mayinclude an avatar, a photo, a moving image, a drawing, and/or acharacter, but are not limited thereto according to an exemplaryembodiment.

FIG. 4 is a table for describing a relationship between a plurality ofpieces of object information and a plurality of object images, which arestored in the device 100, according to an exemplary embodiment.

Referring to FIG. 4, each piece of object information included in anobject field has a corresponding object image. The device 100 may readan object image from a memory of the device 100 by using objectinformation as ID information. When the plurality of pieces of objectinformation and the object images of FIG. 4 stored in an external deviceconnected to the device 100, the device 100 may receive an object imagefrom the external device by using object information.

The device 100 may generate and display an avatar-based image like theone shown in FIG. 1 by combining generated object images. Referring toFIG. 4, the avatar-based image shown in FIG. 1 may be an image generatedby combining a female avatar corresponding to the female, an objectimage corresponding to the first trip place, and an object imagecorresponding to the first preparation.

Returning to FIG. 2, in operation S202, the device 100 may receive auser input indicating a request to change an avatar-based imagedisplayed as shown in FIG. 1. Examples of the user input may include atouch-based user input, a voice-based user input, and a gesture-baseduser input, but the user input according to an exemplary embodiment isnot limited thereto.

FIGS. 5A through 5E are diagrams for describing an example of the device100 changing an avatar-based image, according to an exemplaryembodiment. In FIGS. 5A through 5E, preparations for a trip are changed.

Referring to FIG. 5A, when a user input selecting the information 106indicating that the picture 104 of the bag is a changeable image isreceived, the device 100 displays images 501 through 504 respectivelyregarding first through fourth preparations together with theavatar-based image as shown in FIG. 5B. If the first preparation is thebag, the device 100 may display images 502 through 504 respectivelyregarding second through fourth preparations. If the first preparationis the bag and the device 100 display images 501 through 504respectively regarding the first through fourth preparations, the device100 may display the same image as the picture 104 of the bag, as animage 501 regarding the first preparation. The device 100 may displaythe image 501 regarding the first preparation with a smaller displaysize than a display size of the picture 104 of the bag. Also, the device100 may display a change button 505, an add button 506, and/or acomplete button 507.

In FIG. 5B, when a user input selecting the change button 505 isreceived, the device 100 changes display states of the images 501through 504 respectively regarding the first through fourth preparationsas shown in FIG. 5C. Accordingly, the user recognizes that the firstthrough fourth preparations are changeable.

When a user input selecting the image 501 of the first preparation inFIG. 5C is received, the device 100 displays detail information aboutthe first preparation as shown in FIG. 5D.

Referring to FIG. 5D, the detail information about the first preparationmay be provided by displaying a popup window 508 near the image 501 ofthe first preparation, but an exemplary embodiment is not limitedthereto. Since the detail information provided through the popup window508 is detail information about the first preparation, the user maydelete or change the first preparation while looking at the detailinformation through the popup window 508. The device 100 may display thedetail information in a different display region instead of in a form ofthe popup window 508. For example, the detail information may bedisplayed through a region where the duration information 101 isdisplayed.

When the user selects the popup window 508 displaying the detailinformation as shown in FIG. 5E, the device 100 may display furtherdetail information about the first preparation. The device 100 mayprovide the further detail information by using another page. When thefurther detail information is provided by using the other page, thedevice 100 may change a screen of FIG. 5E to a page providing thefurther detail information.

FIGS. 6A through 6E are diagrams for describing another example of thedevice 100 changing an avatar-based image, according to an exemplaryembodiment. In FIGS. 6A through 6E, a route of a trip is changed.

Referring to FIG. 6A, when a user input selecting the information 105 isreceived, the device 100 displays images 601 through 605 indicatingfirst through fifth trip places and information (for example, arrows)indicating an order of visiting the first through fifth trip places,together with an avatar-based image. The images 601 through 605 mayinclude text information. The text information may include a trip placename. The images 601 through 605 may include a symbol, a mark, an icon,or a thumbnail of a representative image of a trip place.

Referring to FIG. 6B, the device 100 displays the image 601 indicatingthe first trip place in a manner to distinguish the image 601 from theimages 602 through 605 indicating the second through fifth trip placessuch that a user easily recognizes that the photo 103 about a trip placeincluded in the avatar-based image corresponds to the first trip place.For example, in FIG. 6B, the device 100 may highlight the image 601indicating the first trip place.

The device 100 may display, together with the avatar-based image, achange button 606, an add button 607, and/or a complete button 608. Thecomplete button 608 is a button for receiving a user input completing achange operation and/or an add operation performed on the avatar-basedimage displayed on the device 100.

When a user input selecting the change button 606 is received in FIG.6B, the device 100 changes display states of the images 601 through 605as shown in FIG. 6C. Accordingly, the user easily recognizes that theimages 601 through 605 are changeable.

In FIG. 6D, when a user input moving the image 601 to the image 602 isreceived, the device 100 changes the photo 103 based on the first tripplace to a photo 609 based on the second trip place as shown in FIG. 6Dwhile changing an arrangement of the images 601 through 605 as shown inFIG. 6E. Accordingly, the user recognizes that the route is changed suchthat the second trip place is visited first.

With reference again to FIG. 2, when the user selects the completebutton 608 after checking the changed route as shown in FIG. 6E, thedevice 100 updates the trip schedule information according to theavatar-based image in operation S203, such that the trip scheduleinformation is changed to an order of the original second trip place,the original first trip place, the third trip place, the fourth tripplace, and the fifth trip place as shown in FIG. 6E.

FIG. 7 is a flowchart of a method of managing user information, whereinthe device 100 displays an avatar-based image according to user scheduleinformation, and updates the user schedule information by addinginformation based on the displayed avatar-based image, according to anexemplary embodiment.

Referring to FIG. 7, in operation S701, the device 100 displays anavatar-based image according to user schedule information. The device100 may display the avatar-based image according to the user scheduleinformation as described above with reference to operation S201.

In operation S702, when a user input indicating addition is received,the device 100 displays an input window. The device 100 may display theinput window as an independent window.

When information is received through the input window, the device 100adds an image corresponding to the received information to theavatar-based image, in operation S703. For example, when the receivedinformation is a passport, the device 100 may add an image correspondingto the passport to the avatar-based image. That is, in the case of apassport, the user may enter the word “passport” through the inputwindow, and may then be prompted to add an image of the passport. Forexample, the passport image may be received by the user scanning in thepassport into the device 100, or may be received by the user taking apicture of the passport with the device 100. Additionally, oralternatively, the device 100 may pre-store images that may correspondto received information. Images that may correspond to the receivedinformation may be stored in an external device connectable to thedevice 100. A search keyword or search ID information may be assigned toeach image corresponding to the received information. The device 100 maysearch for an image corresponding to the received information by usingthe search keyword or the search ID information. For example, in thecase of the passport, the passport image may be stored on an externaldevice and the passport image may be searched for by the device 100.

The device 100 may search for and provide at least one candidate image(or a plurality of candidate images) corresponding to the receivedinformation whenever one letter (or one alphabet) is received. Forexample, at least one candidate image corresponding to the receivedinformation when the alphabet “p” is received may be different or thesame as at least one candidate image corresponding to the receivedinformation when the alphabet “pa” is received. When a signal ofselecting one candidate image is received, the device 100 may add theselected candidate image to the avatar-based image.

In operation S704, the device 100 updates the user schedule informationaccording to the avatar-based image to which the image is added.

FIGS. 8A through 8C are diagrams for describing another example of thedevice 100 changing an avatar-based image, according to an exemplaryembodiment. In FIGS. 8A through 8C, the device 100 adds a preparation.

When a user input indicating information addition based on the addbutton 506 is received in FIG. 8A, the device 100 displays an inputwindow 801 as shown in FIG. 8B. When information about a fifthpreparation is input through the input window 801 and a user inputselecting an input button 802 is received, the device 100 adds an image803 indicating the fifth preparation to the avatar-based image as shownin FIG. 8C.

The device 100 may display the image 803 distinguishably from the images501 through 504. Accordingly, the user easily recognizes that the image803 corresponding to the fifth preparation is added.

When a user input selecting the complete button 507 is received in FIG.8C, the device 100 updates the user schedule information according tothe avatar-based image to which the image 803 is added. Accordingly, theimages 501 through 504 corresponding to the first through fourthpreparations included in the user schedule information are changed tothe images 501 through 504 and 803 corresponding to the first throughfifth preparations. Thus, user may then be prompted to input an imagefor display with the avatar that corresponds to the fifth preparation.

FIG. 9 is a flowchart of a method of managing user information, whereinthe device 100 displays a plurality of avatar-based images according touser schedule information of a certain period, and updating the userschedule information according to change in at least one of theplurality of avatar-based images, according to an exemplary embodiment.

In operation S901, the device 100 receives a user input indicating arequest for user schedule information of a certain period. The certainperiod may be set based on date information. The certain period may beset based on days. The certain period may be set based on a week unit.The certain period may be set based on time (i.e. a temporal period).However, the certain period according to an exemplary embodiment is notlimited thereto.

In operation S902, the device 100 displays a plurality of avatar-basedimages according to weekday and weekend user schedule information. Here,a weekday and a weekend are a weekday and a weekend included in thecertain period. For example, when the certain period is from Feb. 02,2015 to Feb. 15, 2015, the device 100 may display an avatar-based imageaccording to weekday user schedule information from Feb. 02 to Feb. 06,an avatar-based image according to weekend user schedule informationfrom Feb. 07 to Feb. 08, an avatar-based image according to weekday userschedule information from Feb. 09 to Feb. 13, and an avatar-based imageaccording to weekend user schedule information from Feb. 14 to Feb. 15.

When a user input indicating change is received in operation S903 whilethe plurality of avatar-based images are displayed, the device 100changes at least one of the plurality of avatar-based images.

In operation S904, the device 100 may update the user scheduleinformation according to the changed at least one of the plurality ofavatar-based images.

FIG. 10 illustrates an example of the device 100 displaying anavatar-based image 1001 based on weekday user schedule information andan avatar-based image 1002 based on weekend user schedule information,according to an exemplary embodiment.

FIG. 11 is a flowchart of a method of managing user information, whereinthe device 100 displays a plurality of avatar-based images according touser schedule information per theme item based on a request for userschedule information of a certain period, and updates the user scheduleinformation by changing at least one of the plurality of avatar-basedimages, according to an exemplary embodiment.

Referring to FIG. 11, upon receiving a user input indicating a requestfor user schedule information of a certain period in operation S1101,the device 100 displays a plurality of avatar-based images according touser schedule information per theme item in operation S1102. Here, thetheme item is included in the user schedule information of the certainperiod.

In operation S1103, the device 100 changes at least one of the pluralityof avatar-based images according to a user input indicating a change. Inoperation S1104, the device 100 updates the user schedule informationaccording to the changed at least one of the plurality of avatar-basedimages.

FIGS. 12A through 12C are diagrams for describing an example ofdisplaying a plurality of avatar-based images per theme item included inthe user schedule information of the certain period of FIG. 11, andchanging at least one of the plurality of avatar-based images, accordingto an exemplary embodiment.

Referring to FIG. 12A, the device 100 displays a plurality ofavatar-based images 1201 through 1204 per theme item from Feb. 01 toFeb. 8, 2015.

When a user input selecting the avatar-based image 1201 is received asshown in FIG. 12B from among the plurality of avatar-based images 1201through 1204 of FIG. 12A, the device 100 may enlarge and display theavatar-based image 1201 on an entire screen of the device 100 as shownin FIG. 12C.

Referring to FIG. 12C, the avatar-based image 1201 may include a changebutton 1205, an add button 1206, and/or a complete button 1206. In FIG.12C, a range of change may include changing bike riding to another themeitem (or another user activity) or canceling the bike riding, but therange of change according to an exemplary embodiment is not limitedthereto. For example, when the bike riding is changed to no schedule,the device 100 may change the avatar-based image 1201 to theavatar-based image 1204 of FIG. 12A.

A range of options for addition in FIG. 12C may include adding a datetheme item with B to the bike riding. In this case, the device 100 mayadd an image related to B to the avatar-based image 1201. The range ofoptions for addition according to an exemplary embodiment is not limitedthereto.

FIG. 13 is a flowchart of a method of managing user information, whereinthe device 100 manages a life of an avatar-based image, according to anexemplary embodiment.

In operation S1301, when an offering period is set with respect to anavatar-based image, the device 100 checks the offering period. Theoffering period may be counted based on dates. The offering period maybe counted based on time. The offering period may be counted based onthe number of times being provided. A basis of counting the offeringperiod is not limited thereto according to an exemplary embodiment.

It is then determined in operation S1302 whether the offering period hasexpired. When it is determined that the offering period is expired(S1302, YES), the device 100 removes the avatar-based image in operationS1303. The avatar-based image may be removed by deleting theavatar-based image stored in the device 100. The avatar-based image maybe stored in an external device connected to the device 100. When theavatar-based image is stored in the external device, the device 100 maytransmit a command to delete the avatar-based image to the externaldevice.

FIG. 14 is a flowchart of a method of managing user information, whereinthe device 100 displays an avatar-based image according to healthinformation of a user, and updates user schedule information accordingto selected medical information, according to an exemplary embodiment.

In operation S1401, the device 100 displays an avatar-based imageaccording to health information of a user. The health information may bereceived from an external device, but an exemplary embodiment is notlimited thereto. The health information may be obtained by executing ahealth application installed in the device 100.

In operation S1402, the device 100 provides medical information relatedto the health information according to a user input indicating a medicalinformation request. For example, when the health information indicatesthat the user has a cold, the device 100 may provide medical informationrelated to a cold.

In operation S1403, the device 100 updates user schedule informationaccording to medical information selected based on the provided medicalinformation.

FIGS. 15A through 15E are diagrams for describing an example of thedevice 100 displaying an avatar-based image according to healthinformation of a user, providing medical information according to thehealth information, and updating user schedule information according toselected medical information, according to an exemplary embodiment.

In FIG. 15A, the device 100 displays an avatar-based image with a feverand a runny nose according to health information of a user received froma smart bracelet 1500.

When a user input indicating a medical information request is input, thedevice 100 provides menu items including a hospital item 1511, apharmacy item 1512, a drug item 1513, and a self-treatment item 1514 asshown in FIG. 15B. For example, when a user input touching theavatar-based image of FIG. 15A for a threshold period of time isreceived, the device 100 may provide the menu items of FIG. 15B. Themenu items provided according to an exemplary embodiment is not limitedthereto.

When a user input selecting the hospital item 1511 is received in FIG.15B, the device 100 provides sub-items of the hospital item 1511, suchas a B hospital 1521, a C hospital 1522, and a D hospital 1523, as shownin FIG. 15C. For example, when a user input touching the hospital item1511 for a threshold period of time is received in FIG. 15B, the device100 may provide the sub-items of the hospital item 1511, i.e., the Bhospital 1521, the C hospital 1522, and the D hospital 1523, as shown inFIG. 15C.

When a user input selecting the C hospital 1522 is received as shown inFIG. 15D, the device 100 provides appointment information regarding theC hospital 1522 as shown in FIG. 15E. For example, when a user inputtouching the C hospital 1522 for a threshold period of time is receivedin FIG. 15D, the device 100 may provide the appointment informationregarding the C hospital 1522 in FIG. 15E.

When a user input indicating an appointment is received after a userchecks the appointment information in FIG. 15E, the device 100 performsa procedure for making an appointment in the C hospital 1522. Forexample, when a user input touching an appointment button 1532 for athreshold period of time is received after the user checks theappointment information in FIG. 15E, the device 100 may perform theprocedure for making an appointment. The procedure is performed bymaking an appointment in the C hospital 1522 based on the appointmentinformation and user information. Accordingly, a server of the Chospital 1522 may store the appointment information about the user

When the procedure is completed, the device 100 updates user scheduleinformation based on the appointment information about the user in the Chospital 1522. Accordingly, the appointment information in the Chospital 1522 may be included (or registered) in the user scheduleinformation stored in the device 100.

FIG. 16 is a flowchart of a method of managing user information, whereinthe device 100 displays an avatar-based image according to future hopeinformation of a user and updates user schedule information according toinformation selected from information related to the future hopeinformation, according to an exemplary embodiment.

In operation S1601, the device 100 displays an avatar-based imageaccording to future hope information of a user. The future hopeinformation may indicate future job information of the user, but is notlimited thereto according to an exemplary embodiment. For example, thefuture hope information may include a bucket list or a wish list of theuser.

In operation S1602, the device 100 provides information related to thefuture hope information according to a user input requesting theinformation related to the future hope information. For example, when auser input touching, with a finger, the avatar-based image for athreshold period of time and then dragging the finger in one directionis received, the device 100 may provide the information related to thefuture hope information.

In operation S1603, the device 100 updates user schedule informationaccording to information selected from the information related to thefuture hope information. For example, when a user input touching, with afinger, one piece of the information related to the future hopeinformation for a threshold period of time and then dragging the fingerin one direction is received, the device 100 may update the userschedule information according to the touched piece of informationrelated to the future hope information.

FIGS. 17A through 17D are diagrams for describing processes of thedevice 100 displaying an avatar-based image according to future hopeinformation of a user, providing information related to the future hopeinformation according to a request, and updating user scheduleinformation based on information selected by the user, according to anexemplary embodiment.

Referring to FIG. 17A, the device 100 displays an avatar-based imageindicating an art therapist according to future hope information of auser.

When a user input 1701 of requesting for information related to thefuture hope information is received as shown in FIG. 17A, the device 100provides, as the information related to the future hope information, aqualifying examination schedule item 1711, a certification center item1712, and/or a related event item 1713 as shown in FIG. 17B. Forexample, when the user input 1701 is received by touching, with afinger, an image displayed in FIG. 17A for a threshold period of timeand dragging the finger in one direction, the device 100 providesinformation related to the future hope information, such as the qualityexamination schedule item 1711, the certification center item 1712,and/or the related event item 1713, as shown in FIG. 17B.

When a user input selecting the related event item 1713 is received inFIG. 17B, the device 100 provides a first event item 1721, a secondevent item 1722, and a third event item 1723. For example, when a userinput touching the related event item 1713 for a threshold period oftime is received in FIG. 17B, the device 100 may provide the first eventitem 1721, the second event item 1722, and the third event item 1723,which are registered in a sub-layer of the related event item 1713.

In FIG. 17C, when a user input selecting the second event item 1722 isreceived, the device 100 provides detail information about the secondevent item 1722. For example, when a user input touching the secondevent item 1722 for a threshold period of time is received in FIG. 17C,the device 100 may provide the detail information about the second eventitem 1722.

The device 100 may provide the detail information about the second eventitem 1722 by opening a separate window 1724 near the second event item1722. The separate window 1724 may be displayed as a popup window havinga shape of a speech bubble, but is not limited thereto.

In FIG. 17D, when a user input selecting a participate OK button 1725included in the separate window 1724 is received, the device 100 updatesuser schedule information by using information related to the secondevent item 1722. For example, in FIG. 17D, when a user input touchingthe participate OK button 1725 included in the separate window 1724 fora threshold period of time is received, the device 100 may update theuser schedule information by using the information related to the secondevent item 1722. Accordingly, the information related to the secondevent item 1722 is included in the user schedule information.

FIG. 18 is a flowchart of a method of managing user information based ona plurality of avatar-based images according to usage historyinformation of the device 100, according to an exemplary embodiment.

When a user input requesting usage history information of the device 100is received in operation S1801, the device 100 displays a plurality ofavatar-based images according to the usage history information per usageperiod of the device 100 in operation S1802. For example, when a userinput indicating selection on a usage history information request itemof the device 100 included in menu items provided by the device 100 isreceived in operation S1801, the device 100 may display a plurality ofavatar-based images according to usage history information per usageperiod in operation S1802. In operation S1801, information specifying ausage period of the device 100 may be received. The informationspecifying a usage period may be input based on a date, time, and/or aday of the week.

FIGS. 19A and 19B are diagrams for describing an example of providing,by the device 100, usage history information per usage period in aplurality of avatar-based images upon receiving a user input indicatinga usage history information request of the device 100, according to anexemplary embodiment.

Referring to FIG. 19A, the device 100 displays a plurality ofavatar-based images according to usage history information of the device100 regarding weekdays 1910 and weekends 1920.

Referring to the weekdays 1910 of FIG. 19A, the device 100 uses a phonecall application the most and an Internet application the second most onthe weekdays 1910. Referring to the weekends 1920 of FIG. 19A, thedevice 100 uses a digital television (DTV) application the most and aBand application the second most on the weekends 1920.

In FIG. 19A, when a user input 1911 of requesting for next usage historyinformation of the device 100 on the weekdays 1910 is received, thedevice 100 may change an avatar-based image of the weekdays 1910 asshown in FIG. 19B based on the next usage history information of thedevice 100 on the weekdays 1910. Referring to the weekdays 1910 of FIG.19B, the device 100 uses a Kakao Talk application the third most and acamera application the fourth most on the weekdays 1910.

In FIGS. 19A and 19B, the usage history information of the device 100 isprovided based on applications installed in the device 100, but is notlimited thereto according to an exemplary embodiment.

FIG. 20 is a flowchart of a method of managing user information, whereinthe device 100 displays an avatar-based image according to userinformation registered in social media, changes the displayedavatar-based image according to a user input, and updates the userinformation registered in the social media according to the changedavatar-based image, according to an exemplary embodiment.

In operation S2001, the device 100 displays an avatar-based imageaccording to user information registered in social media. The userinformation registered in the social media may be received from a socialmedia server connected to the device 100. For example, user informationregistered in Facebook may be received from a Facebook server connectedto the device 100.

The user information registered in the social media may be provided fromthe social media server according to a request of the device 100, oralternatively, may be automatically provided from the social mediaserver when the device 100 is connected to the social media server.According to an exemplary embodiment, a method of receiving, by thedevice 100, the user information registered in the social media is notlimited thereto.

In operation S2002, the device 100 changes the avatar-based imageaccording to a user input requesting to change the avatar-based image.The avatar-based image may be changed in operation S2002 in the samemanner described above with reference to operation S202 of FIG. 2.

In operation S2003, the device 100 may update the user informationregistered in the social media according to the changed avatar-basedimage. The device 100 may convert the changed avatar-based image to textinformation, and request the social media server to update the userinformation registered in the social media while transmitting the textinformation to the social media server. Accordingly, the social mediaserver may update the user information. After the user information isupdated, the social media server may transmit an update complete messageto the device 100. According to an exemplary embodiment, a method ofupdating the user information registered in the social media is notlimited thereto.

FIG. 21 is a block diagram of a system 2100 for managing userinformation, which includes the device 100, according to an exemplaryembodiment.

Referring to FIG. 21, the system 2100 includes the device 100, a network2110, and an external device 2120.

The device 100 may include, for example, a portable device, a wearabledevice, a home Internet appliance, or a mobile device, but the device100 according to an exemplary embodiment is not limited thereto.

The portable device may be a smart phone, a notebook, a smart board, atablet PC, a handheld device, a handheld computer, a media player, anelectronic book device, or a personal digital assistant (PDA), but theportable device according to an exemplary embodiment is not limitedthereto.

The wearable device may include a device, such as smart glasses, a smartwatch, a smart band (for example, a smart waist band or a smart hairband), a smart clip, a smart accessory (for example, a smart ring, asmart bracelet, a smart anklet, a smart hairpin, or a smart necklace), asmart body guard (for example, a smart knee guard or a smart elbowguard), a smart shoe, a smart glove, smart clothing, a smart hat, asmart artificial leg for a disabled person, or a smart artificial armfor a disabled person, but is not limited thereto.

The home Internet appliance may include a home device forming machine tomachine (M2M) or Internet of Things (IoT), such as a smart TV, a PC, adesktop PC, a smart refrigerator, a smart washing machine, a smart lamp,a smart cleaner, a smart fitness equipment, a smart oven, a smart dishwasher, a multimedia console, a game console, or a smart water purifier,but the home Internet appliance according to an exemplary embodiment isnot limited thereto.

The mobile device may include a device, such as a car, which forms M2Mor IoT, but the mobile device according to an exemplary embodiment isnot limited thereto.

The network 2110 may include, for example, Intranet, Internetwork,Internet, a personal area network (PAN) (for example, a near fieldwireless communication network such as Bluetooth or home radio frequency(HomeRF)), a campus area network (CAN), a local area network (LAN), awide area network (WAN), a game network, a Wi-Fi communication, or aWi-Fi direct communication, but the network 2110 according to anexemplary embodiment is not limited thereto.

The network 2110 may use at least one transmission medium from among awired transmission medium and a wireless transmission medium. The wiredtransmission medium may include, for example, at least one of a coaxialcable, a twisted pair cable, and/or a fiber-optic, but the wiredtransmission medium according to an exemplary embodiment is not limitedthereto. The wireless transmission medium may include, for example, atleast one of radio frequency waves, terrestrial microwaves, satellitemicrowaves, and/or infrared waves, but the wireless transmission mediumaccording to an exemplary embodiment is not limited thereto.

A connection type of the network 2110 may be a star type, a net type, atree type, a bus type, or a ring type, but is not limited thereto.

The network 2110 may use at least one of network connection devices,such as an LAN card, a hub, a switch, a bridge, a gateway, a repeater,and/or a router, but the network connection devices are not limitedthereto.

The external device 2120 is a device disposed outside the device 100 andconnectable to the device 100. The external device 2120 may include acloud server of a user, a user information managing server, a questionand answer (QA) server, a service providing server (for example, a gameserver or an application server), or a social media server, but theexternal device 2120 according to an exemplary embodiment is not limitedthereto.

The external device 2120 may include a server, a medical device, acomputing device, an appcessory, a wearable device, an IoT sensor, or anIoT device. The external device 2120 may be the same device as thedevice 100. A user of the external device 2120 may be the same as a userof the device 100. The user of the external device 2120 may be differentfrom the user of the device 100.

The medical device may have a communication function and a function ofsensing (or measuring) health information of a user. The medical devicemay include a home medical device (for example, a thermometer, a bloodpressure gauge, a cholesterol gauge, a blood sugar gauge, a uric acidgauge, or a massager), a hospital medical device (for example, anultrasonic device, a body fat measurer, a stethoscope, or anelectrocardiogram (ECG) measurer), or a rehabilitation medical device(for example, a gait belt, a moving belt, a knee pad, or a paraffinbathtub), but the medical device is not limited thereto.

The appcessory is a device controlled by using an application installedin the device 100. For example, the appcessory includes a smart bulb, asmart printer, a smart wireless blood pressure gauge, a smart recorder,a smart positioning device, a smart microphone, a smart thermostat, asmart alarm device, or a smart toy.

The device 100 may perform a method of managing user information,according to one or more exemplary embodiment. The device 100 may manageuser information stored in the device 100 and/or user information storedin the external device 2120 connected through the network 2110. The userinformation stored in the device 100 may be received from the externaldevice 2120 and stored in the device 100. The user information stored inthe external device 2120 may be received from the device 100 and storedin the external device 2120. The user information stored in the device100 may be the same as the user information stored in the externaldevice 2120. The user information stored in the device 100 may bedifferent from the user information stored in the external device 2120.A part of the user information stored in the device 100 may be the sameas the user information stored in the external device 2120.

The device 100 may update the user information stored in the device 100and/or the external device 2120 by using user information changed via amethod of managing user information, according to one or more exemplaryembodiment. The device 100 may transmit the user information stored inthe device 100 to the external device 2120 and request the externaldevice 2120 to manage the user information.

FIG. 22 is a flowchart of a method of managing, by the device 100, userinformation, according to an exemplary embodiment. FIGS. 23A through 23Fare diagrams for describing avatar-based images displayed by the device100 based on user information, according to exemplary embodiments.

Referring to FIG. 22, in operation S2201, the device 100 displays anavatar-based image based on user information. For example, the device100 may generate and display an avatar-based image 2310 based on userinformation 2300 of FIG. 23A. The avatar-based image 2310 is an imagesymbolizing a user of the device 100. The avatar-based image 2310 may berepresented in a 2D or 3D model avatar, a cyber character, a virtualcharacter of the user, a virtual self-graphic icon of the user, or auser icon. According to an exemplary embodiment, a method ofrepresenting the avatar-based image 2310 is not limited thereto.

In operation S2201, the device 100 may display a plurality ofavatar-based images based on user information, as shown in FIGS. 23Bthrough 23F. (Operations S2202 and S2203 are described in detail laterfollowing the description of FIGS. 31A-31D.)

Referring to FIG. 23B, the device 100 displays an avatar-based image2340 based on current user information (hereinafter, referred to as acurrent image 2340), and an avatar-based image 2341 based on future userinformation (hereinafter, referred to as a future image 2341).

The current image 2340 and the future image 2341 have different hairstyles. Such different hair styles indicate that hair style information(for example, a hair style of the user is wavy hair) included in thecurrent user information and hair style information (for example, a hairstyle of the user is short hair) included in the future user informationare different.

The hair style information included in the current user information maybe based on photo information, but an exemplary embodiment is notlimited thereto. For example, the hair style information included in thecurrent user information may be input by the user. The hair styleinformation included in the future user information may be based onbeauty parlor appointment information of the user, but an exemplaryembodiment is not limited thereto. For example, the hair styleinformation included in the future user information may be input by theuser.

The device 100 may display the current image 2340 and the future image2341 by dividing user information in units of a day, units of severaldays, units of a week, units of a month, or units of a year.

For example, when the user information is divided in units of a day, thedevice 100 may generate and display the current image 2340 based ontoday's user information (for example, today's schedule information ofthe user), and generate and display the future image 2341 based ontomorrow's user information (for example, tomorrow's scheduleinformation of the user).

For example, when the user information is divided in units of a week,the device 100 may generate and display the current image 2340 based onuser information of the past week including that day, and generate anddisplay the future image 2341 based on user information of the next weekincluding the following day.

The current image 2340 and the future image 2341 displayed on the device100 may be the same. In this case, the current user information and thefuture user information are the same.

The device 100 may generate the current image 2340 and the future image2341 in other ways than those described above. For example, the device100 may generate the current image 2340 in a school uniform and thefuture image 2341 in a police uniform based on information about acurrent job (for example, a student) and information about a dream job(for example, a policeman), which are included in the user information.

Alternatively, the device 100 may generate and display the future image2341 based on a bucket list or a wish list included in the userinformation. For example, when ten items are included in the bucketlist, the device may generate and display ten avatar based images basedon the ten items as ten future images. For example, when the ten futureimages are generated and displayed based on the ten items, the device100 may divide an area where the future images 2341 are displayed tosimultaneously display the ten future images, or may display the tenfuture images in ten pages, but an exemplary embodiment is not limitedthereto.

In FIG. 23C, the device 100 according to an exemplary embodimentsimultaneously displays an avatar based image 2342 based on past userinformation (hereinafter, referred to as a past image 2342), the currentimage 2340, and the future image 2341.

Referring to FIG. 23C, the device 100 simultaneously displays the pastimage 2342, the current image 2340, and the future image 2341 havingdifferent hair styles. Such different hair styles indicate that hairstyle information included in the past user information, the currentuser information, and the future user information are different fromeach other.

As described above with reference to FIG. 23B, the device 100 maysimultaneously generate and display the past image 2342, the currentimage 2340, and the future image 2341 based on user information in unitsof a day, several days, a week, a month, or a year. For example, thedevice 100 may generate the current image 2340 based on user informationfor a past week including user information of current time based on thecurrent time, generate the future image 2341 based on user informationfor a week after the current time, and generate the past image 2342based on user information for a past week based on the user informationused to provide the current image 2340.

In FIG. 23D, the device 100 according to an exemplary embodimentsimultaneously displays an avatar-based image based on morning userinformation (hereinafter, referred to as a morning image 2343) and anavatar-based image based on afternoon user information (hereinafter,referred to as an afternoon image 2344). The device 100 may generate themorning image 2343 and the afternoon image 2344 based on units of a day,several days, a week, a month, or a year, but the units of generatingthe morning image 2343 and the afternoon image 2344 are not limitedthereto.

Referring to FIG. 23D, the morning image 2343 indicates that the userinformation includes information “go to work in the morning”. Theafternoon image 2344 indicates that the user information includesinformation “have violin lessons after work”.

In FIG. 23E, the device 100 according to an exemplary embodimentsimultaneously displays an avatar-based image 2345 based on weekday userinformation (hereinafter, referred to as a weekday image 2345) and anavatar-based image 2346 based on weekend user information (hereinafter,referred to as a weekend image 2346).

Referring to FIG. 23E, the device 100 generates the weekday image 2345based on user information from Monday to Friday. Also, the device 100may generate the weekend image 2346 based on user information ofSaturday and Sunday. The device 100 may generate the weekday image 2345and the weekend image 2346 based on units of a week, several weeks, amonth, a year, or several years, but the units are not limited thereto.

The weekday image 2345 indicates that the user information includesinformation “go to work from Monday to Friday”. The weekend image 2346indicates that the user information includes information “cycle onweekends”. According to an exemplary embodiment, the weekday image 2345and the weekend image 2346 displayed on the device 100 are not limitedto those shown in FIG. 23E.

In FIG. 23F, the device 100 according to an exemplary embodimentsimultaneously displays an avatar-based image 2347 based on userinformation in spring (hereinafter, referred to as a spring image 2347),an avatar-based image 2348 based on user information in summer(hereinafter, referred to as a summer image 2348), an avatar-based image2349 based on user information in autumn (hereinafter, referred to as anautumn image 2349), and an avatar-based image 2350 based on userinformation in winter (hereinafter, referred to as a winter image 2350).

The device 100 may generate the spring image 2347 based on userinformation, for example, from March to May. The device 100 may generatethe summer image 2348 based on user information, for example, from Juneto August. The device 100 may generate the autumn image 2349 based onuser information, for example, from September to November. The device100 may generate the winter image 2350 based on user information, forexample, from December to February.

The spring image 2347 and the autumn image 2349 indicate that the userinformation includes information “go to work by bicycle”. The summerimage 2348 and the winter image 2350 indicate that the user informationincludes information “go to work by car”. The device 100 may generateand display the spring image 2347, the summer image 2349, the autumnimage 2349, and the winter image 2350, which are different from eachother according to the user information.

The device 100 may generate and display the spring image 2347, thesummer image 2348, the autumn image 2349, and the winter image 2350 asmonthly images or images in units of several months.

The device 100 may generate and display the avatar-based image 2310, thecurrent image 2340, the future image 2341, the past image 2342, themorning image 2343, the afternoon image 2344, the weekday image 2345,the weekend image 2346, the spring image 2347, the summer image 2348,the autumn image 2349, and/or the winter image 2350 of FIGS. 23A through23F by including various objects thereto according to the userinformation.

In the one or more exemplary embodiments described above, the device 100generating and displaying an avatar-based image means that the device100 provides the avatar-based image.

Hereinafter, for convenience of description, it is assumed that anavatar-based image displayed on the device 100 is the avatar-based image2310 of FIG. 23A. However, an avatar-based image according to anexemplary embodiment is not limited thereto. For example, theavatar-based image 2310 displayed on the device 100 may be any one ofthe current image 2340, the future image 2341, the past image 2342, themorning image 2343, the afternoon image 2344, the weekday image 2345,the weekend image 2346, the spring image 2347, the summer image 2348,the autumn image 2349, and/or the winter image 2350 of FIGS. 23A through23F according to the user information. Also, the avatar-based image 2310displayed on the device 100 may be an image obtained by adding at leastone object to one of the current image 2340, the future image 2341, thepast image 2342, the morning image 2343, the afternoon image 2344, theweekday image 2345, the weekend image 2346, the spring image 2347, thesummer image 2348, the autumn image 2349, and the winter image 2350 ofFIGS. 23A through 23F according to the user information.

The device 100 may represent the avatar-based image 2310 in at least oneof a static image and a dynamic image. For example, the device 100 mayrepresent the avatar-based image 2310 in a dynamic image and after afirst time, represent the avatar-based image 2310 in a static image.Alternatively, the device 100 may represent the avatar-based image 2310in a static image and after a second time, represent the avatar-basedimage 2310 in a dynamic image. The first or second times may be pre-set.The first and second times may be the same or different from each other.

Referring to FIG. 23A, the user information 2300 includes hobbyinformation 2301 of the user, name 2302 of the user, and genderinformation 2303 of the user. According to an exemplary embodiment, theuser information 2300 is not limited to that shown in FIG. 23A.

For example, the user information 2300 may further include at least onepiece of the user information described above. The user information 2300may include fewer information than those shown in FIG. 23A. For example,the user information 2300 may include the hobby information 2301 of theuser and the name 2302 of the user.

In order to generate the avatar-based image 2310, the device 100 maygenerate user modeling information regarding the user information 2300.The user modeling information may be referred to as a user modelinginformation table.

The device 100 may generate the user modeling information regarding theuser information 2300 based on ontology. A method of generating the usermodeling information regarding the user information 2300 is not limitedto ontology.

FIG. 24A illustrates an example of the user modeling information 2400regarding the user information 2300, according to an exemplaryembodiment, which is based on ontology. The ontology has been describedabove with reference to FIG. 3

Referring to FIG. 24A, the user modeling information 2400 includes asubject field 2401, a predicate field 2402, and an object field 2403.

The subject field 2401 of FIG. 24A is a field related to a subject ofthe user information 2300. Referring to FIG. 24A, the subject of theuser information 2300 is the first user User_1. In FIG. 24A, a class2402 and a resource 2405 are used to indicate that the first user User_1is a person. The resource 2405 defines an instance of the class 2404.

The predicate field 2402 of FIG. 24A is a field indicating a formallanguage of a relationship between words included in the subject field2401 and the object field 2403. Accordingly, the predicate field 2402 ofFIG. 24A includes a formal language (Name) about a name, a formallanguage (Gender) about a gender, and a formal language (hasHobby) abouta hobby.

According to the formal language about a name, “A” included in theobject field 2403 may be represented as a name of the first user User_1.According to the formal language about a gender, “FEMALE” included inthe object field 2403 may be represented as a gender of the first userUser_1. According to the formal language about a hobby, “Violin”included in the object field 2403 may be represented as a hobby of thefirst user User_1. Information included in the object field 2403 of FIG.24A is user information of the first user User_1 corresponding to theformal languages included in the predicate field 2402.

The user modeling information 2400 of FIG. 24A is generated in a RDFlanguage. However, the device 100 may generate ontology-based usermodeling information regarding the user information 2300 by using alanguage, such as OWL or SWRL.

FIG. 24B illustrates an example of display sizes of an object includedin the avatar-based image 2310 displayed on the device 100, according toan exemplary embodiment. The example of FIG. 24B is about a violinincluded in the user modeling information 2400 of FIG. 24A.

Referring to FIG. 24B, the device 100 may determine a display size of anobject according to a possible weight value assigned to the object. Asshown in FIG. 24B, the device 100 may determine a display size of aviolin to be included in the avatar-based image 2310 according to apossible weight value assigned to a violin included in the object field2403. The possible weight value may be determined according to userinformation. The display size according to the possible weight value maybe pre-set.

Referring to FIG. 24B, a range of the possible weight value is from 0to 1. An interest level of a user regarding a violin is low when thepossible weight value is 0.3 compared to when the possible weight valueis 1. For example, when it is determined that a hobby of the user isplaying a violin and the user plays a violin every other day for onehour based on user information of the device 100, the device 100 may setthe possible weight value of the violin to 0.5. Alternatively, when itis determined that the hobby of the user is playing a violin and theuser plays a violin every day for one hour based on the user informationof the device 100, the device 100 may set the possible weight value ofthe violin to 1.

When the possible weight value of the violin is 0.5, the device 100determines the display size of the violin to be 50×50 pixels.Accordingly, the device 100 may generate the avatar-based image 2310including an image of the violin in 50×50 pixels.

Alternatively, when the possible weight value of the violin is 1, thedevice 100 determines the display size of the violin to be 100×100pixels. Accordingly, the device 100 may generate the avatar-based image2310 including an image of the violin in 100×100 pixels.

FIG. 24C is a diagram for describing a display size of an objectincluded in the avatar-based image 2310, according to an exemplaryembodiment. The example of FIG. 24C is about a female included in theuser modeling information 2400 of FIG. 24A.

Referring to FIG. 24C, the device 100 may determine a display size of afemale avatar 2311 of FIG. 23A to be included in the avatar-based image2310 according to a weight and a height. The weight and the height arethose of a female included in the object field 2403 of the user modelinginformation 2400. The weight and height are included in the userinformation. The device 100 may select one of a plurality of displaysizes that are pre-set.

Referring to FIG. 24C, the device 100 set a baseline of the weight to 50kg and a baseline of a height to 170 cm. However, the baselinesaccording to an exemplary embodiment are not limited thereto. When aweight of the user included in the user information satisfies a“(height−100)×0.9+5>weight” condition, the device 100 may display thefemale avatar 2311 included in the avatar-based image 2310 in a fatimage.

Alternatively, when the weight of the user satisfies a“(height−100)×0.9<weight” condition, the device 100 may display thefemale avatar 2311 included in the avatar-based image 2310 in a thinimage. When the weight of the user satisfies an “else” condition, thedevice 100 may display the female avatar 2311 included in theavatar-based image 2310 in a normal image.

FIG. 25 illustrates an example of mapping information 2500 between anobject image and object ID information, according to an exemplaryembodiment. Referring to FIG. 25, the device 100 may generate theavatar-based image 2310 by using mapping information 2500 and the usermodeling information 2400 of FIG. 24A. A display size of the objectimage of FIG. 25 may be adjusted or changed based on informationincluded in user information as described above with reference to FIGS.24B and 24C.

The mapping information 2500 may be configured by considering all orsome of the user information described above. The mapping information2500 may be referred to as a mapping information table.

According to an exemplary embodiment, the mapping information 2500 maybe set differently from that shown in FIG. 25. For example, the mappinginformation 2500 may be configured to provide an object image in a stillimage as shown in FIG. 25, or to provide an object image in a movingimage. The mapping information 2500 may be stored in the device 100 orin the external device 2120.

An image corresponding to a violin or a guitar may include a standardimage of playing a violin or a guitar. The standard image may also beprovided in a still image or a moving image. The standard image mayinclude a musical instrument and an avatar image playing the musicalinstrument.

When the standard image is stored in the mapping information 2500, thedevice 100 may change the avatar playing the musical instrument to anavatar of the user. The device 100 may change the avatar playing themusical instrument to the avatar of the user by mapping locationinformation of feature points for displaying the avatar playing themusical instrument and location information of feature points fordisplaying the avatar of the user. Location information of featurepoints may be represented in (x,y) coordinate values when an avatar is a2D image. Location information of feature points may be represented in(x,y,z) coordinate values when an avatar is a 3D image.

FIG. 26 is a diagram for describing processes of generating theavatar-based image 2310 by using the user modeling information 2400 andthe mapping information 2500, according to an exemplary embodiment.

Referring to FIG. 26, the device 100 detects information (A, a female,and a violin) included in the object field 2403 of the user modelinginformation 2400. The device 100 detects object images 2610 and 2620from the mapping information 2500 by using object ID information (forexample, “0000 0010” for the female and “0000 0011” for the violin)assigned to the detected information. The device 100 combines the objectimages 2610 and 2620.

While combining the object images 2610 and 2620, the device 100 may usecoordinate values (2D coordinate values or 3D coordinate values) used torender the object images 2610 and 2620 and pre-set combining conditioninformation.

Coordinate values of object images may be configured in a coordinatevalue table such that a coordinate value may be detected by using objectID information included in the mapping information 2500. For example,the device 100 may detect a coordinate value used to render the objectimage 2610 from the coordinate value table by using “0000 0010”. Thecoordinate value table may include location information of featurepoints for displaying each object image. The coordinate value table maybe stored in the device 100 and/or the external device 2120.

The coordinate value table may be a skeleton data table of an objectimage. For example, when an object image is an avatar, such as theobject image 2610, the skeleton data table shows an outline of theavatar. For example, the skeleton data table of the avatar may includeinformation about a face, a head, a neck, a shoulder, a chest, a waist,a hip, a length of an arm, a thickness of an arm, a length of a leg, athickness of a leg, a right hand and/or a left hand, but is not limitedthereto.

For example, when an object image is a violin, such as the object image2620, the skeleton data table shows an outline of the violin. Forexample, the skeleton data table of the violin may include informationabout a scroll, a peg, a peg box, a nut, a neck, a fingerboard, astring, a soundboard, a purfling, a side plate, a waist, a bridge, anf-hole, a tail-piece, a chin rest, and/or a button, but is not limitedthereto.

Combining condition information may include location information ofobject images to be combined. The combining condition information may beconfigured in a combining condition information table such that thecombining condition information is detected by using object IDinformation of object images to be combined. For example, when theobject images 2610 and 2620 are combined as shown in FIG. 26, the device100 may detect combining condition information between the object images2610 and 2620 from the combining condition information table by usingthe object ID information of the object images 2610 and 2620, i.e.,“0000 0010” and “0000 0011”.

When the combining condition information includes, for example, locationinformation indicating “the object image 2610 holds a bow with a righthand and holds a neck of a violin with a left hand”, the device 100 mayprovide an image 2630 by combining the object image 2610 and the objectimage 2620. Location information indicates location information offeature points for displaying each object. The combining conditioninformation table may be pre-stored in the device 100 and/or theexternal device 2120.

In order to generate the avatar-based image 2310, the combiningcondition information may include location information indicating “holda bow with a right hand, put a chin on a chin rest, and hold a neck of aviolin with a left hand”. When the object image 2620 of a violin is astandard image of playing a violin, the device 100 may change coordinatevalues (or location information of feature points) used to render aviolinist playing a violin to coordinate values of the object image2610.

Referring back to FIG. 22, in operation S2002, the device 100 changesthe avatar-based image 2310 according to a user input. The user inputmay be a user input indicating a change request with respect to theavatar-based image 2310, a user input indicating object selection, or auser input indicating change completion, but is not limited thereto. Theuser input indicating the change request may be a user input indicatingan edit request. The user input indicating the change completion may bea user input indicating edit completion.

The user input may be based on at least one of touch, touch and drag, agesture, multi-modal, and voice. The user input may be received from theexternal device 2120.

When the user input indicating the change request is received while theavatar-based image 2310 is displayed, the device 100 may display achangeable object.

FIGS. 27A through 27F illustrate examples of the device 100 displaying achangeable object, according to exemplary embodiments.

Referring to FIG. 27A, when a user input selecting a change requestbutton 2701 is received, the device 100 highlights a name 2702 of a userand a violin 2703. Accordingly, the user recognizes that the name 2702and the violin 2703 are changeable objects.

In FIG. 27B, when a user input selecting a change request button 2705 isreceived, the device 100 may overlap and display a message 2706indicating a name in a partial region of the name 2702 of the user.Also, the device 100 overlaps and displays a message 2707 indicating ahobby in a partial region of the violin 2703. The message 2707 mayinclude information related to the violin 2703, for example, playing aviolin. Accordingly, the user recognizes that the name 2702 and theviolin 2703 are changeable objects, or that the message 2706 indicatingthe name and the message 2707 indicating a hobby are changeable objects.

Referring to FIG. 27C, the device 100 may display, together with theavatar-based image 2310, a tray 2712 including icons 2710 and 2711regarding changeable objects.

In FIG. 27C, when a user input selecting a change request button 2709 isreceived, the device 100 provides the tray 2712 including the icon 2710regarding a hobby and the icon 2711 regarding a name. The icon 2710 mayinclude information related to the violin 2703. Accordingly, the usermay recognize that the name 2702 and the violin 2703 or the name 2702and a hobby item are changeable objects. The tray 2712 may furtherinclude ID information (hobby and name) of the icons 2710 and 2711.

Referring to FIG. 27C, the device 100 displays the tray 2712 on thebottom of a screen. Alternatively, the tray 2712 may be displayed on atleast one of the left, the right, the top, or the bottom of the screen.When the number of icons of changeable objects is greater than thenumber of icons of objects simultaneously displayable on the tray 2712,the device 100 may display icons of selectable objects while allowingthe icons displayed through the tray 2712 to be moved according to ascroll input based on the tray 2712. The scroll input based on the tray2712 may include a scroll input based on a region of the tray 2712.

When the number of icons of changeable objects is greater than thenumber of icons of objects simultaneously displayable on the tray 2712,the device 100 may provide a plurality of trays classified according topre-set categories.

When the device 100 provides the plurality of trays, the device 100 mayarrange one of the plurality of trays on at least one of the left, theright, the top, or the bottom of the screen, or arrange the plurality oftrays in a hierarchical structure on at least one of the left, theright, the top, or the bottom of the screen. A category may be setaccording to a type of a changeable object, but is not limited thereto.

In FIG. 27C, when an icon of an object included in the tray 2712 isselected, the device 100 may highlight the object of the selected icon.

In FIG. 27D, the device 100 according to an exemplary embodimentprovides information about a changeable object through a speech bubbleaccording to a touch-based input.

In FIG. 27D, when a user input selecting a change request button 2714 isreceived, the device 100 may activate a changeable image from amongdisplayed images.

When a touch-based input 2715 indicating selection on the changeableobject (for example, the violin 2703) from among the displayed images isreceived, the device 100 displays a speech bubble 2716 includingchangeable information (for example, a hobby) regarding the changeableobject (the violin 2703). The device 100 may display the speech bubble2716 near the violin 2703 to indicate that the speech bubble 2716 isrelated to the violin 2703. Accordingly, the user may recognize that theviolin 2703 is the changeable object and user information about thehobby is changeable.

Referring to FIG. 27E, the device 100 provides the changeable object ina form of a popup window 2721. In FIG. 27E, when a user input indicatingselection on a change request button 2718 is received, the device 100displays the popup window 2721 including information 2719 and 2720 aboutchangeable objects. The popup window 2721 may include information aboutat least one object.

Accordingly, the user recognizes the information 2719 and 2720 about thechangeable objects. When information about one object included in thepopup window 2721 is selected, the device 100 may highlight the objectcorresponding to the selected information.

Referring to FIG. 27F, the device 100 may display a changeable object byusing an independent page. In FIG. 27F, when a user input selecting achange request button 2723 is received, the device 100 opens a page 2726including information 2724 and 2725 about changeable objects.Accordingly, the user recognizes the changeable objects. The device 100may display information about at least one changeable object through thepage 2726.

When a changeable object is selected in FIG. 27E or 27F, the device 100may highlight the selected changeable object.

In FIGS. 27A through 27F, the user input indicating the selection on thechange request button 2701, 2705, 2709, 2714, 2718, or 2723 arereceived, but the user input indicating the change request are notlimited thereto according to an exemplary embodiment. For example, thedevice 100 may receive the user input by using another button or basedon a user gesture.

FIG. 28 is a flowchart of a method of changing, by the device 100, theavatar-based image 2310 according to a user input, according to anexemplary embodiment.

Referring to FIG. 28, in operation S2801, the device 100 receives afirst selection signal regarding one object from among changeableobjects. In operation S2802, the device 100 displays candidate objectsof the selected object.

FIGS. 29A through 29F illustrate examples of changing an object based oncandidate objects provided by the device 100, according to exemplaryembodiments.

When the device 100 receives the touch-based input 2715 as the firstselection signal indicating selection on the violin 2703 as shown in ascreen 2920 while highlighting a changeable object as shown in a screen2910 of FIG. 29A, the device 100 displays candidate objects related tothe violin 2703.

Referring to the screen 2920 of FIG. 29A, the device 100 provides aguitar item 2912, a baseball item 2913, and a badminton item 2914 as thecandidate objects related to the violin 2703, but the candidate objectsaccording to an exemplary embodiment are not limited thereto.

Referring to the screen 2920 of FIG. 29A, the device 100 displaysinformation about candidate objects in a form of a popup window 2921.Referring to the screen 2920 of FIG. 29A, the device 100 provides theinformation about the candidate objects in text.

However, a method of providing the information about the candidateobject, according to an exemplary embodiment is not limited thereto. Forexample, as shown in a screen 2930 of FIG. 29B, the device 100 mayprovide the information about the candidate objects in a form of arectangular popup window 2931. The rectangular popup window 2931arranges images respectively corresponding to the candidate objectshorizontally. The rectangular popup window 2931 may provide theinformation about the candidate objects in symbols, icons, orrepresentative images, but the information about the candidate objectsis not limited thereto.

As shown in a screen 2940 of FIG. 29B, the device 100 may provide theinformation about the candidate objects in a form of a ring type popupwindow 2941. The ring type popup window 2941 may include the informationabout the candidate objects in symbols or text.

When the number of candidate objects is higher than the number ofobjects simultaneously displayable on the popup window 2921, 2931, or2941, the device 100 may provide the information about the candidateobjects while moving the information about the candidate objectsdisplayed on the popup window 2921, 2931, or 2941 according to a scrollinput based on the popup window 291, 2931 or 2941. The device 100 maymove the information about the candidate objects according to a pre-setarrangement order.

When the number of candidate objects is higher than the number ofobjects simultaneously displayable on the popup window 2921, 2931, or2941, the device 100 may provide the information about the candidateobjects by expanding a display area of the popup windows 2921, 2931, or2941 according to an input (for example, touch and drag) based on thepopup window 2921, 2931, or 2941.

For example, when the user executes a touch and drag input on screen2930 shown in FIG. 29B, the screen 2950 of FIG. 29C is displayed. Thatis, when the user executes the touch and drag input on screen 2930, thedevice 100 may provide the information about the candidate objects byexpanding the rectangular popup window 2931 according to a touch anddrag input 2951. Alternatively, as shown in a screen 2960 of FIG. 29D,the device 100 may further provide a popup window 2961 separately fromthe rectangular popup window 2931 according to the touch and drag input2951.

Returning to FIG. 28, in operation S2803 of FIG. 28, the device 100receives a second selection signal based on information about thecandidate objects. In operation S2804, the device 100 changes a selectedobject to the candidate object selected according to the secondselection signal.

When the second selection signal of selecting the baseball item 2913 isreceived in a screen 2970 of FIG. 29E, the device 100 changes anavatar-based image of playing the violin 2703 to an avatar-based imagewith a baseball glove on a left hand and a ball on a right hand as shownin a screen 2980.

The avatar-based image of playing the baseball shown in the screen 2980according to an exemplary embodiment is not limited thereto. Forexample, when a user input selecting the baseball item 2913 is receivedwhile displaying an avatar-based image holding the violin 2703 with aleft hand as shown in a screen 2990 of FIG. 29F, the device may displayan avatar-based image holding a baseball bat 2996 with a left hand.Alternatively, when a user input selecting a movie item 2915 is receivedthrough a popup window 2911′, the device 100 may display an avatar-basedimage holding a movie CD or a video tape with a left hand.

When a user input selecting a change complete button 2704 is received inthe screen 2980 of FIG. 29E or a screen 2995 of FIG. 29F, the device 100may generate and display a changed image 2320 similar to that shown inFIG. 23A. The changed image 2320 of FIG. 23A is an avatar-based imageplaying baseball.

FIG. 30 is a flowchart of a method of providing, by the device 100,service information providable according to objects included in theavatar-based image 2310, according to an exemplary embodiment.

Referring to FIG. 30, in operation S3001, the device 100 receives afirst selection signal indicating selection on an object. In operationS3002, the device 100 provides at least one piece of service informationbased on the selected object.

FIGS. 31A through 31D illustrate examples of a screen for describing themethod of FIG. 30. When a user input selecting the change request button2701 is received while an avatar-based image is displayed as shown in ascreen 3110 of FIG. 31A, the device 100 highlights a changeable objectas shown in a screen 3120 of FIG. 31A.

When a first selection signal is received with respect to the violin2703 in the screen 3120 of FIG. 31A, the device 100 provides providableservice information based on the violin 2703 through a popup window 3131as shown in a screen 3130 of FIG. 31B. Referring to the popup window3131, the providable service information may include a concertreservation item, a community access item and/or a specialized shoppingwebsite item, but the providable service information according to anexemplary embodiment is not limited thereto.

With reference to FIG. 30 and FIGS. 31A through 31D, when a secondselection signal 3132 indicating selection on the selected object (theviolin 2703 of screen 3130 of FIG. 13B) is received in operation S3003,the device 100 provides information about at least one candidate objectregarding the selected object (the violin 2703) in operation S3004. Theinformation about the at least one candidate object may be provided in aform of a popup window 3141 as shown in a screen 3140 of FIG. 31B.

Accordingly, the device 100 determines the first selection signalreceived in operation S3001 as preliminary selection regarding an objectthe user wishes to change, and the second selection signal received inoperation S3003 as completion selection regarding the object the userwishes to change. The preliminary selection indicates a selectionstandby state regarding the object, and the completion selectionindicates a selection completion state regarding the object.

In operation S3005, the device 100 receives a third selection signalbased on the information about the at least one candidate object.Accordingly, in operation S3006, the device 100 provides at least onepiece of service information regarding the selected candidate object.

Referring to FIG. 31C, when a baseball item is selected from theinformation about the at least one candidate object displayed on thepopup window 3141 of a screen 3150, the device 100 provides providableservice information based on the baseball item through a speech bubble3151 as shown in a screen 3160.

In operation S3007, the device 100 receives a fourth selection signalregarding the selected candidate object (the baseball item). Forexample, double touches 3152 of the screen 3160 of FIG. 31C may bereceived as the fourth selection signal.

The device 100 may determine the third selection signal regarding theselected candidate object received in operation S3005 as preliminaryselection regarding the selected candidate object and the fourthselection signal regarding the selected candidate object received inoperation S3007 as completion selection regarding the selected candidateobject. The preliminary selection indicates a selection standby stateand the completion selection indicates a selection completion state.

In operation S3008, the device 100 changes the selected object to theselected candidate object. Accordingly, the device 100 may display anavatar-based image based on the baseball item as shown in a screen 3170of FIG. 31D. When a user input selecting the change complete button 2704is received in the screen 3170 of FIG. 31D, the device 100 completes animage change.

According to the method of FIG. 30, the user of the device 100 may checkprovidable service information based on a candidate object beforechanging an avatar-based image.

Returning again to FIG. 22, when the avatar-based image 2310 is changedaccording to the user input in operation S2202 of FIG. 22, the device100 updates the user information 2300 to user information 2330 based onthe changed image 2320 of FIG. 23A in operation S2203. The userinformation 2330 includes hobby information 2331 (playing a guitar) ofthe user, a name 2332 of the user, and gender information 2333 (female)of the user.

By updating the user information 2300 to the user information 2330, theuser modeling information of FIG. 24A may be updated to user modelinginformation 3200 of FIG. 32. FIG. 32 illustrates an example of the usermodeling information 3200 updated by the device 100, according to anexemplary embodiment.

When a display size of the female avatar 2311 included in theavatar-based image 2310 is changed, the device 100 may change userinformation according to the changed display size. For example, when thedisplay size of the female avatar 2311 included in the avatar-basedimage 2310 is changed from normal to thin according to a user input, thedevice 100 may update a weight and a height of the user included in theuser information to values that express thinness.

While the avatar-based image 2310 is displayed by the device 100 inoperation S2201 of FIG. 22, the device 100 may also provide sound basedon the user information 2300. The sound may be provided as backgroundmusic of the avatar-based image 2310 or voice of the female avatar 2311.The voice of the female avatar 2311 may provide, for example, guideinformation related to the user information 2300 (for example, “I am A,female, and my hobby is playing a violin”.)

The device 100 may provide the voice of the female avatar 2311 in anyone of various modes based on a female voice. Examples of the variousmodes may include a speech mode, a musical mode, and/or a play mode, butare not limited thereto. Such a mode may be set as a default in thedevice 100, or may be set or changed by the user.

When the sound is provided as background music, the device 100 mayprovide violin music based on the hobby information 2301 (playing aviolin) included in the user information 2300. The violin music may bedetermined by the device 100 or the external device 2120 based on userinformation other than the user information 2300 of FIG. 23A.

The device 100 may further provide, together with the avatar-based image2310, a background image of the female avatar 2311 based on the userinformation 2300. The background image may be a stage image of playing aviolin based on the hobby information 2301 or a single color image (forexample, pink) based on the gender information 2303, but is not limitedthereto. For example, when it is determined that the user usually playsa violin 2312 at night based on user information, the device 100 maydisplay a night image as the background image. The device 100 maydetermine that the user usually plays the violin 2312 at night based ona violin play time included in user information.

As described above, when at least one of the background image and thesound are further provided together with the avatar-based image 2310,the device 100 may provide the background image and the sound aschangeable objects.

When the background image and the sound are provided by user informationother than the user information 2300, and the background image and/orthe sound are changed, the device 100 may update the user informationbased on the changed background image and/or the changed sound.

FIG. 33 is a flowchart of a method of managing, by the device 100, userinformation by changing avatar-based image, according to an exemplaryembodiment. In FIG. 33, the user information is updated according to auser input accepting the changed avatar-based image.

In operation S3301 of FIG. 33, the device 100 displays the avatar-basedimage 2310 based on user information as in operation S2201 describedabove. In operation S3302, the device 100 changes the avatar-based image2310 according to a user input as in operation S2202 described above.

In operation S3303, the device 100 displays the changed image 2320. Itis then determined whether the changed image is acceptable in operationS3304. When a user input accepting the changed image 2320 is received(S3304, YES), operation S3305 is performed. The user input accepting thechanged image 2320 may be input through the device 100, or by using theexternal device 2120. In order to receive the user input accepting thechanged image 2320, the device 100 may display a message asking the userto accept the changed image 2320.

When the user input accepting the changed image 2320 is not received(S3304, NO), operation S3301 is performed and the device 100 may displaythe avatar-based image 2310.

Alternatively, when the user input accepting the changed image 2320 isnot received (S3304, NO), operation S3303 is performed and the device100 may display the changed image 2320 while maintaining a standby stateregarding a user input requesting to additionally change the changedimage 2320.

When a user input is not received for a pre-set period of time inoperation S3304, the device 100 may determine that the user inputaccepting the changed image 2320 is not received.

When the user input accepting the changed image 2320 is received (S3304,YES), the device 100 updates the user information based on the changedimage 2320 in operation S3305.

FIG. 34 is a flowchart of a method of managing, by the device 100, userinformation by changing avatar-based image, according to anotherexemplary embodiment.

Referring to FIG. 34, in operation S3401, the device 100 receives userinformation from the external device 2120. The device 100 may receivethe user information from the external device 2120 according to a userinput requesting the user information. For example, the device 100 mayrequest the external device 2120 for the user information according toan input of requesting user information registered in a social mediaserver, and accordingly, receive the user information from the externaldevice 2120.

Alternatively, the device 100 may request the user information from theexternal device 2120 without any request. For example, when the device100 is in a sleep mode or executing an application A, health informationof the user may be received from the external device 2120. The userinformation received in operation S3401 may include at least one pieceof user information described above. The received user information maybe stored in the device 100.

In operation S3402, the device 100 may generate and display anavatar-based image based on the user information. The avatar-based imagemay be generated based on the user information as described above withreference to operation S2201 of FIG. 22.

In operation S3403, the device 100 changes the avatar-based imageaccording to a user input indicating a change request. The avatar-basedimage may be changed according to the user input as described above withreference to operation S2202 of FIG. 22

In operation S3404, the device 100 updates the user informationaccording to the changed avatar-based image. The device 100 changes theuser information received in operation S3401 according to the changedavatar-based image, and request the external device 2120 to update theuser information while transmitting the changed user information to theexternal device 2120. Accordingly, the external device 2120 may updatethe user information. After requesting the external device 2120 toupdate the user information, the device 100 may automatically delete thestored user information upon.

FIG. 35 is a flowchart of a method of managing user information, whereinthe device 100 changes and/or adds information by using an avatar-basedimage, according to an exemplary embodiment.

In operation S3501, the device 100 displays the avatar-based image 2310according to user information. Operation S3501 may be performed in thesimilar manner as operation S2201 of FIG. 22 described above.

In operation S3502, the device 100 may change the avatar-based image2301 according to a user input. The avatar-based image 2310 may bechanged in operation S3502 in the similar manner as operation S2202 ofFIG. 22.

Also, in operation S3502, the device 100 may add new information (or anew object) to the avatar-based image 2310 according to a user input.The new information may be added based on image informationcorresponding to addable information provided by the device 100. Theimage information may include at least one of an image, a symbol, and anicon symbolizing the addable information.

When the image information corresponding to the addable information isselected, the device 100 may provide the changed image 2320 to which theselected image information is added. However, a process of adding newinformation is not limited thereto.

For example, when image information corresponding to addable information(for example, an appcessory) is selected, the device 100 may provide aninput window for receiving information related to the selected imageinformation, for example, an earring. Upon receiving the informationrelated to the selected image information through the input window, thedevice 100 may generate an image of the received information (forexample, the earring), and provide the changed image 2320 including thegenerated image.

In operation S3503, the device 100 updates user information based on thechanged image 2320. For example, when the fifth preparation is added asdescribed above with reference to FIGS. 8A through 8C, the device 100may update user information such that information about the fifthpreparation is included in the user information.

FIG. 36 is a flowchart of a method of managing user information suchthat the device 100 is able to provide a service based on an objectselected from an avatar-based image, according to an exemplaryembodiment. In FIG. 36, the object is an image and/or text correspondingto changeable user information included in the avatar-based image.

In operation S3601, the device 100 displays avatar-based image accordingto user information. The device 100 may display the avatar-based imagein the same manner as in operation S2201 of FIG. 22 described above. Forexample, the device 100 may display the avatar-based image of FIG. 31A.At this time, the device 100 may display the avatar-based image thatdoes not include the change request button 2701 and the change completebutton 2704 of FIG. 31A.

In operation S3602, the device 100 provides service information based onan object, for example, the violin 2703 upon receiving a user inputselecting the violin 2703 from the screen 3110. For example, when theuser input selecting the violin 2703 is received as in the screen 3120of FIG. 31A, the device 100 opens the popup window 3131 including atleast one piece of service information based on the violin 2703 as shownin the screen 3130 of FIG. 31B.

Referring to FIG. 31B, the service information included in the popupwindow 3131 includes the concert reservation item, the community accessitem, and the specialized shopping website item. However, the serviceinformation included in the popup window 3131 is not limited thereto.Also, the service information based on the violin 2703 according to anexemplary embodiment may be provided in a form of the tray 2712 of FIG.27C. The service information based on the violin 2703 may referred to asa service information based on an object selected by a user input.

It is then determined whether service information is selected inoperation S3603. When a user input selecting one piece of serviceinformation is received (S3603, YES), the device 100 provides theselected service information in operation S3604. For example, when auser input selecting the concert reservation item included in the popupwindow 3131 is received, the device 100 may execute an applicationproviding a violin concert reservation service.

FIG. 37 is a flowchart of a method of managing user information, whereinthe device 100 provides a question and answer (QA) service based on anavatar-based image being displayed, according to an exemplaryembodiment.

In operation S3701 of FIG. 37, the device 100 displays an avatar-basedimage according to user information. For example, the device 100 maydisplay the avatar-based image 2310 of FIG. 23A according to userinformation.

It is then determined whether question information is received inoperation S3702. When question information is received (S3702, YES), thedevice 100 provides answer information regarding the questioninformation in operation S3703.

The question information received in operation S3702 is questioninformation related to the avatar-based image. For example, the questioninformation may be “tell me information about the violin concert Ireserved”, “tell me if I have a violin lesson tomorrow”, “what time wasthe violin lesson yesterday?”, or “what is my schedule immediatelybefore a violin lesson tomorrow?”.

The question information received in operation S3702 may use a voicesignal of the user. If the device 100 is unable to process (orrecognize) the question information received in operation S3702, thedevice 100 may display an error message or output a voice signalindicating an error message.

Also, if the device 100 fails to recognize the question information, thedevice 100 may provide guide information regarding inputtable questioninformation. The device 100 may provide the guide information regardingthe inputtable question information in a form of a list. The guideinformation may provide information about a form of the inputtablequestion information.

While recognizing the question information, the device 100 may considerthe user information. For example, when the question information is“give me information about Lincoln”, the device 100 may determinewhether “Lincoln” is a person or a car based on the user information. Ifthe device 100 determines that the user is highly interested in carsbased on the user information, the device 100 may recognize “Lincoln”included in the question information as a car “Lincoln”.

Meanwhile, if the question information received in operation S3702 is“tell me information about the violin concert I reserved”, the device100 may provide a screen 3810 or 3820 of FIG. 38 in operation S3703. Forexample, if the question information received in operation S3702 is“tell me information about the concert I recently updated”, the device100 may provide the screen 3810 before an update and the screen 3820after the update of FIG. 38 together. Accordingly the user may easilycheck information about concerts before and after an update.

FIG. 38 illustrates an example of the device 100 providing answerinformation according to received question information, according to anexemplary embodiment. Referring to FIG. 38, answer information based onuser information before an update and answer information based on theuser information after the update are displayed at the same time.

The device 100 may partially overlap and display the screens 3810 and3820. The device 100 may provide the screens 3810 and 3820 in a similarmanner to an electronic book being opened. When the screens 3810 and3820 are provided in such a manner, and a user input touching anddragging a corner or a side of the screen 3810 or 3820 is received, thedevice 100 may close the screens 3810 and 3820 in a dragging direction.For example, when a user input touching and dragging, in a leftwarddirection, a rightmost side of the screen 3820 is received, the device100 may close the screens 3810 and 3820 in such a manner that the screen3820 is turned over to cover the screen 3810.

FIG. 39 is a flowchart of a method of managing user information based onthe device 100 and the external device 2120, according to an exemplaryembodiment. In FIG. 39, the external device 2120 generates anavatar-based image.

In operation S3901, the external device 2120 generates an avatar-basedimage based on user information. The external device 2120 may generatethe avatar-based image based on the user information in the same manneras in operation S2201 of FIG. 22 described above. For example, when theexternal device 2120 is a social media server, the external device 2120may generate an avatar-based image based on user profile informationregistered in the social media server.

In operation S3902, the external device 2120 transmits the avatar-basedimage to the device 100. Operation S3902 may be performed according to arequest of the device 100 for user information.

In operation S3903, the device 100 displays the avatar-based image. Inoperation S3904, the device 100 changes the avatar-based image.Operation S3904 may be performed by receiving a user input indicating arequest to change the avatar-based image. The avatar-based image may bechanged in the same manner as in operation S2202 of FIG. 22 describedabove.

When the avatar-based image is changed, the device 100 transmits thechanged avatar-based image to the external device 2120 in operationS3905. In operation S3906, the external device 2120 updates the userinformation based on the changed avatar-based image. Operation S3906 maybe performed in the same manner as in operation S2203 of FIG. 22.

FIG. 40 is a flowchart of a method of managing user information based onthe device 100 and the external device 2120, according to anotherexemplary embodiment. In FIG. 40, the device 100 generates anavatar-based image.

When user information is transmitted from the external device 2120 tothe device 100 in operation S4001, the device 100 generates anavatar-based image based on the user information in operation S4002.

The device 100 displays the avatar-based image in operation S4003. Thedevice 100 changes the avatar-based image according to a user input inoperation S4004.

In operation S4005, the device 100 updates the user information based onthe changed avatar-based image. In operation S4006, the device 100transmits the updated user information to the external device 2120. Inoperation S4007, the external device 2120 may update the userinformation stored in the external device 2120 based on the receivedupdated user information.

FIG. 41 is a flowchart of a method of managing user information based onthe device 100 and a wearable device 4110, according to an exemplaryembodiment. In FIG. 41, information about an object included in anavatar-based image is displayed through the wearable device 4110.

The device 100 displays an avatar-based image according to userinformation in operation S4101, and transmits information about achangeable object included in the avatar-based image to the wearabledevice 4110 in operation S4102.

In operation S4103, the wearable device 4100 displays the informationabout the changeable object. According to an exemplary embodiment, theinformation about the changeable object may be displayed in a form of animage, but is not limited thereto. For example, the wearable device 4110may display the information about the changeable object in a form oftext. When a signal of selecting an object based on the informationabout the changeable object is received in operation S4104, the wearabledevice 4110 transmits information about the selected object to thedevice 100 in operation S4105.

In operation S4106, the device 100 searches stored information forinformation about a candidate object of the selected object. Theinformation about the candidate object may be searched for from anexternal storage apparatus of the device 100. The information about thecandidate object may be searched for from the external device 2120connected to the device 100. The device 100 may search for theinformation about the candidate object by using ID information of theselected object as a keyword.

In operation S4107, the device 100 transmits the information about thecandidate object to the wearable device 4110. In operation S4108, thewearable device 4110 displays the information about the candidateobject.

For example, when the object selected in operation S4104 is the violin2703 of FIG. 29A, the information about the candidate object, which maybe displayed in operation S4108, may include items included in the popupwindow 2921 of the screen 2920 of FIG. 29A.

For example, when the object selected in operation S4104 is the icon2710 regarding a hobby of FIG. 27C, the information about the candidateobject, which may be displayed in operation S4108, may include items(for example, swimming, bike riding, and photographing) related to ahobby.

The information about the candidate object may be displayed in a form ofan image and/or text, but is not limited thereto.

When a signal of selecting a candidate object is received in operationS4109, the wearable device 4110 transmits information about the selectedcandidate object to the device 100 in operation S4110.

In operation S4111, the device 100 changes the avatar-based image byusing the selected candidate object. In operation S4112, the device 100updates the user information based on the changed avatar-based image.The user information may be updated in operation S4112 in the samemanner as in operation S2203 of FIG. 22 described above.

FIG. 42 is a flowchart of a method of managing user information based onthe device 100 and a wearable device 4210, according to anotherexemplary embodiment. In FIG. 42, an avatar-based image is changed basedon user information obtained by the wearable device 4210.

In operation S4201, the device 100 displays an avatar-based imageaccording to user information. In operation S4201, the device 100 maydisplay the avatar-based image in the same manner as in operation S2201of FIG. 22 described above.

In operation S4202, the wearable device 4210 obtains the userinformation. The user information obtained by the wearable device 4210may include, for example, health information of a user or locationinformation of the wearable device 4210, but the user informationobtained by the wearable device 4210 is not limited thereto.

Operations S4201 and S4202 may be simultaneously performed.Alternatively, operation S4202 may be performed before operation S4201,or operation S4201 may be performed before operation S4202.

In operation S4203, the wearable device 4210 transmits the userinformation to the device 100. The device 100 changes the avatar-basedimage displayed in operation S4201 based on the received userinformation, in operation S4204.

For example, when the received user information is information about atemperature of the user and the temperature is determined to be higherthan a reference temperature, the device 100 may change a face of theavatar-based image displayed in operation S4201 to be partially red.Accordingly, the user may check that the user has a fever through thechanged avatar-based image displayed through the device 100.

For example, when the received user information is information about ahappy feeling of the user, the device 100 may change a face of theavatar-based image displayed in operation S4201 to a smiling face.Accordingly, the user may check that the user is in a good mood throughthe changed avatar-based image displayed through the device 100.

As described above, the device 100 may variously change the avatar-basedimage displayed in operation S4201 according to the user informationobtained and transmitted by the wearable device 4210. While changing theavatar-based image, not only the face of the avatar-based image, butalso at least one changeable object from among objects included in theavatar-based image may be changed. The changeable object may furtherinclude at least one of a background image and sound, but is not limitedthereto. In operation S4205, the device 100 may update the userinformation based on the changed avatar-based image.

FIG. 43 is functional block diagrams of the device 100 according toexemplary embodiments. Referring to FIG. 43, the device 100 may includea user input module 4310, a controller 4320, a display 4330, and amemory 4340.

The user input module 4310 may receive a user input requesting userinformation, a user input requesting to change user information, userinput requesting to add user information, a user input completingchanging of user information, a user input selecting an object, a userinput requesting service information, a user input selecting serviceinformation, a user input accepting a changed avatar-based image, or auser input controlling the device 100, but a user input according to anexemplary embodiment is not limited thereto.

The user input module 4310 may be mounted on the device 100. The userinput module 4310 mounted on the device 100 may include, for example, atleast one of a keypad, a dome switch, a touch pad (a contact capacitancetype, a pressure resistance film type, an infrared detection type, asurface ultrasound conducting type, an integral tension measuring type,or a piezo-effect type), a jog wheel, and/or a jog switch, but the userinput module 4310 according to an exemplary embodiment is not limitedthereto.

A user input received through the user input module 4310 may be atouch-based user input, a movement-based user input, a vision-based userinput and/or voice-based user input, but the user input is not limitedthereto.

Example of the touch-based user input may include tap (or touch), longtap (or long touch), touch and hold, touch and drag, double tap, drag,panning, flick, drag and drop, sweep, or swipe, but the touch-based userinput is not limited thereto.

Example of the movement-based user input may include user input based onmovement-based user gesture of the device 100 (for example, shaking thedevice 100, rotating the device 100, or lifting the device 100), but themovement-based user input is not limited thereto.

Example of the vision-based user input may include user input recognizedby analyzing input images obtained by a camera included in the device100 without a contact between the device 100 and the user, but thevision-based user input is not limited thereto.

The controller 4320 is one or more processors controlling operations ofthe device 100. The controller 4320 may load and operate a program forexecuting a method of managing user information, according to one ormore exemplary embodiments, which is stored in the memory 4340. Thecontroller 4320 may store the program for executing the method in thememory 4340. The controller 4320 may update the program stored in thememory 4340.

The controller 4320 may generate an avatar-based image based on userinformation according to the method, and display the avatar-based imageon the display 4330. Upon receiving a user input requesting to changethe avatar-based image from the user input unit 4310, the controller4320 changes the avatar-based image according to the user input. Thecontroller 4320 may update the user information based on the changedavatar-based image.

The controller 4320 may control the operations of the device 100 basedon the method of any one of FIGS. 2, 7, 9, 11, 13, 14, 16, 18, 20, 22,28, 30, 33 through 37, 39, and 40 through 42.

Also, when a user input requesting user information of a certain periodis received through the user input module 4310, the controller 4320controls the display 4330 to display a plurality of avatar-based imagesaccording to the user information of the certain period, in response tothe user input. The plurality of avatar-based images displayed on thedisplay 4330 may be determined based on standards of dividing thecertain period.

Also, when a user input requesting user information of a certain periodis received through the user input module 4310, the controller 4320controls the display 4330 to display a plurality of avatar-based imagesaccording to the user information of the certain period, in response tothe user input. The plurality of avatar-based images displayed on thedisplay 4330 may be determined based on theme items included in the userinformation of the certain period.

Also, when a user input requesting to display changeable information isreceived through the user input module 4310, the controller 4320 maydistinguishably display a changeable region and an unchangeable regionof the avatar-based image displayed on the display 4330.

Also, when an offering period is set with respect to the avatar-basedimage, the controller 4320 checks the offering period. When the offeringperiod is expired, the controller 4320 may remove the avatar-basedimage. Information about the offering period and the avatar-based imagemay be stored in the memory 4340, but an exemplary embodiment is notlimited thereto. For example, the information about the offering periodand the avatar-based image may be stored in the external device 2120.

Also, when a user input indicating a medical information request isreceived through the user input module 4310 while the avatar-based imageis displayed on the display 4330 based on health information of a user,the controller 4320 displays medical information related to the healthinformation on the display 4330 in response to the user input.

When a user input selecting medical information from among the medicalinformation provided through the display 4330 is received through theuser input module 4310, the controller 4320 may update user scheduleinformation according to the selected medical information.

Also, when a user input adding information to the avatar-based image isreceived through the user input module 4310, the controller 4320 maydisplay an input window for adding information on the display 4330. Wheninformation is received through the input window, the controller 4320adds an image corresponding to the received information to theavatar-based image displayed on the display 4330. The controller 4320may update the user schedule information according to the avatar-basedimage to which the image is added. The user schedule information may bestored in the memory 4340 and/or the external device 2120.

Also, the controller 4320 may control the display 4330 to display theavatar-based image according to future hope information of the user.When a user input requesting information related to the future hopeinformation is received through the user input module 4310 while theavatar-based image according to the future hope information is displayedon the display 4330, the controller 4320 provides the informationrelated to the future hope information through the display 4330 inresponse to the user input. When a user input selecting at least onepiece of the information related to the future hope information providedthrough the display 4330 is received through the user input module 4310,the controller 4320 may update the user schedule information accordingto the selected information related to the future hope information.

Also, when a user input requesting usage history information per usageperiod of the device 100 is received through the user input module 4310,the controller 4320 may display a plurality of avatar-based imagesaccording to the usage history information on the display 4330 inresponse to the user input.

Also, the controller 4320 may control the display 4330 to display anavatar-based image according to user information registered in socialmedia. Here, when a user input requesting to change is received throughthe user input module 4310, the controller 4320 may change theavatar-based image displayed on the display 4330 in response to the userinput, and update the user information registered in the social mediaaccording to the changed avatar-based image. The user informationregistered in the social media may include user profile informationregistered in a social media server as described above. The userinformation registered in the social media, according to an exemplaryembodiment, is not limited thereto.

The display 4330 may display information processed by the device 100.The display 4330 may display a graphical user interface (GUI) screen asdescribed above. For example, the display 4330 may display a GUI screenas described above with reference to FIGS. 1, 5A through 5E, 6A through6E, 8A through 8C, 10, 12A through 12C, 15A through 15E, 17A through17D, 19A and 19B, 23A through 23F, 27A through 27F, 29A through 29F, 31Athrough 31D, or 38.

When the display 4330 and a touch pad form a layer structure to beconfigured as a touch screen, the display 4330 may be used as an inputdevice and an output device. The display 4330 may include at least oneof a liquid crystal display (LCD), a thin-film transistor-liquid crystaldisplay (TFT-LCD), an organic light-emitting diode (OLED), a flexibledisplay, a 3D display, and an electrophoretic display (EPD). But, thedisplay 4330 is not particularly limited.

According to an exemplary embodiment of the device 100, the device 100may include at least two displays. Here, the at least two displays maybe disposed to face each other by using a hinge. The display may be atransparent display unit.

The memory 4340 may store programs for processes and controls of thecontroller 4320. The programs stored in the memory 4340 may include anoperating system (OS) program and various application programs. Variousapplication programs may include an application for executing at leastone of a method of managing user information and a method of providing aservice according to an avatar-based image, according to one or moreexemplary embodiments, but the application is not limited thereto.

The memory 4340 stores information managed by an application program.For example, the memory 4340 may store user information according to oneor more exemplary embodiments, user modeling information based on userinformation, coordinate values of each object, and/or combiningcondition information of each object, but the information stored in thememory 4340 is not limited thereto.

The memory 4340 may include at least one storage medium from among aflash memory, a hard disk, a multimedia card micro type memory, a cardtype memory (for example, a secure digital (SD) card or an extremedigital (XD) card), a random access memory (RAM), a static random accessmemory (SRAM), a read-only memory (ROM), an electrically erasableprogrammable read-only memory (EEPROM), a programmable read-only memory(PROM), a magnetic memory, a magnetic disk, or an optical disk. But, thememory 4340 is not particularly limited.

The device 100 may be configured to use web storage or a cloud server(not shown), which performs a storage function of the memory 4340 on theInternet. At this time, information about a GUI displayed on the display4330 may be provided from the web storage or the cloud server.

Not all components of the device 100 illustrated in FIG. 43 areessential. The device 100 may include greater or fewer components thanthose shown in FIG. 43. For example, the device 100 may be configured asshown in FIG. 44.

Referring to FIG. 44, the device 100 may include a user interface module4410, a controller 4420, a sensor module 4430, a memory 4440, an imageprocessor 4450, an audio output unit 4460, an audio input unit 4470, acamera 4480, and a communication module 4490.

The device 100 may further include a battery (not shown). The batterymay be embedded in the device 100 or detachably included in the device100. The battery may supply power to all components included in thedevice 100. The device 100 may receive power from an external powersupply device. The external power supply device may be a batteryembedded in the external device 2120.

A user input module 4411 and the display 4412 included in the userinterface module 4410, the controller 4420, and the memory 4440 of FIG.44 are similar to or same as the user input module 4310, the controller4320, the display 4330, and the memory 4340 of FIG. 43.

The sensor module 4430 may include a proximity sensor 4431 detectingproximity of the user to the device 100, an illumination sensor 4432 (oran optical sensor) detecting lighting around the device 100, a Moodscopesensor 4433 detecting a mood of the user of the device 100, a positionsensor 4434 (for example, a GPS receiver) detecting a position of thedevice 100, a gyroscope sensor 4435 measuring an azimuth of the device100, an acceleration sensor 4436 (accelerometer) measuring a tilt andacceleration of the device 100 based on the ground surface, aterrestrial magnetic sensor 4437 detecting azimuth orientation based onthe device 100, a gesture sensor 4438 recognizing a hand gesture bydetecting infrared light reflected from a hand, and/or atemperature/humidity sensor 4439 recognizing ambient temperature andhumidity. When the position sensor 4434 is configured as a GPS receiver,the position sensor 4434 may be included in the communication module4490. The gesture sensor 4438 may be a movement detecting sensordetecting activity.

The sensor module 4430 is not limited to that shown in FIG. 44. Forexample, the sensor module 4430 may further include a gravity sensor, analtitude sensor, a chemical sensor (for example, an odorant sensor), anatmospheric sensor, a fine dust measuring sensor, an infrared lightsensor, an ozone level sensor, a carbon dioxide (CO2) sensor, and/or anetwork sensor (for example, a Wi-Fi, Wi-Fi direct, Bluetooth, 3G, longterm evolution (LTE), or near field communication (NFC)-based networksensor).

The sensor module 4430 may include a pressure sensor (for example, atouch sensor, a piezoelectric sensor, or a physical button), a statussensor (for example, an earphone terminal, a digital multimediabroadcasting (DMB) antenna, or a standard terminal (for example, aterminal for recognizing a charging level, a terminal for recognizingpersonal computer (PC) connection, or a terminal for recognizing dockconnection)), a time sensor, and/or a health sensor (for example, abio-sensor, a heart rate sensor, a blood flow rate sensor, a diabeticsensor, a blood pressure sensor, or a stress sensor), but a sensorincluded in the sensor module 4430 according to an exemplary embodimentis not limited thereto.

A result (or a sensing value or a sensing signal) detected by the sensormodule 4430 is transmitted to the controller 4420.

The controller 4420 may generate and display, on the display 4412, anavatar-based image based on a sensing value received from the sensormodule 4430. The controller 4420 may change the avatar-based imagedisplayed on the display 4412 according to a user input requesting tochange the avatar-based image, which is received through the user inputmodule 4411. The controller 4420 may update user information stored inthe memory 4440 based on the changed avatar-based image. The controller4420 may update user information stored in the external device 2120connected through the communication module 4490, according to thechanged avatar-based image. As such, the controller 4420 may operate inthe same manner as the controller 4320 of FIG. 43, according to anexemplary embodiment.

The controller 4420 controls overall operations of the device 100. Forexample, the controller 4420 may execute programs stored in the memory4440 to control the user interface module 4410, the sensor module 4430,the memory 4440, the image processor 4450, the audio output unit 4460,the audio input unit 4470, the camera 4480, and/or the communicationmodule 4490.

The controller 4420 may control operations of the device 100 based onthe method described above. For example, the controller 4420 may controloperations of the device 100 based on the method described in any one ofFIGS. 2, 7, 9, 11, 13, 14, 16, 18, 20, 22, 28, 30, 33, 34 through 37,and 39 through 42, as the controller 4320 of FIG. 43.

The device 100 may include a plurality of the controllers 4420 accordingto an operation environment. For example, the controller 4420 mayinclude a processor that operates in a low power mode and a processorthat operates in a normal mode of the device 100. The controller 4420may include a plurality of processors according to components includedin the device 100.

Programs stored in the memory 4440 may be classified into a plurality ofmodules according to functions. For example, the programs stored in thememory 4440 may be classified into a UI module 4441, a notificationmodule 4442, and/or an application module 4443, but the programs storedin the memory 4440 according to an exemplary embodiment are not limitedthereto. For example, when the display 4412 is combined with a touchpanel, the plurality of modules may include a touch screen module.

The UI module 4441 may provide GUI information for changing userinformation described above according to an exemplary embodiment to thecontroller 4420. The UI module 4441 may provide a UI or GUI specializedwith respect to each application installed in the device 100 to thecontroller 4200.

The notification module 4442 may generate a signal notifying a requestfor a change, a request for addition, completion of a change, objectselection, request for service information, or service selection of thedevice 100, but a notification generated by the notification module 4442is not limited thereto.

The notification module 4442 may output a notification signal in a formof a video signal through the display 4412, or in a form of an audiosignal through the audio output unit 4460, but is not limited thereto.

The application module 4443 may include at least one of a userinformation managing application set in the device 100 and a serviceproviding application based on an avatar-based image.

The user information managing application according to an exemplaryembodiment may operate by being connected to another application set inthe device 100. For example, when the question information is receivedin operation S3702 of FIG. 37, the user information managing applicationmay be connected to a QA application set in the device 100 so as toprovide answer information (for example, a search result) based on thequestion information, in operation S3703.

The image processor 4450 processes image data received from thecommunication module 4490 or stored in the memory 4440 such as to bedisplayed on the display 4412. For example, the image processor 4450 mayperform a signal process regarding image data such that a GUI screendescribed above with reference to FIGS. 1, 5A through 5E, 6A through 6E,8A through 8C, 10, 12A through 12C, 15A through 15E, 17A through 17D,19A and 19B, 23A through 23F, 27A through 27F, 29A through 29F, 31Athrough 31D, and 38 is displayed on the display 4412.

The audio output unit 4460 may output sound according to userinformation, according to an exemplary embodiment. The audio output unit4460 may output audio data received from the communication module 4490or stored in the memory 4440. The audio output unit 4460 may output asound signal (for example, notification sound) related to a functionperformed by the device 100. The audio output unit 4460 may include aspeaker or a buzzer, but is not limited thereto.

The audio input unit 4470 may be configured as a microphone. The audioinput unit 4470 may receive an audio signal input from an externaldevice of the device 100, convert the audio signal to an electric audiosignal, and transmit the electric audio signal to the controller 4420.The audio input unit 4470 may be configured to perform an operationbased on any one of various noise removing algorithms for removing noisegenerated while receiving an external sound signal.

The camera 4480 may obtain an image frame of a still image or a movingimage through an image sensor, when the device 100 is in a video callmode or a photographing mode. The camera 4480 may obtain and transmit,to the controller 4420, a finger gesture image of the user of the device100. The controller 4420 may determine, based on a finger gesture imageof the user, whether a user input requesting to change an avatar-basedimage, a user input requesting to add information to an avatar-basedimage, a user input completing changing of an avatar-based image, a userinput selecting an object from an avatar-based image, a user inputrequesting to provide service information based on an avatar-basedimage, or a user input requesting to execute a service based on anavatar-based image is received. According to an exemplary embodiment,the finger gesture image is not limitedly used to determine whether theuser input is received. An image captured by an image sensor may beprocessed through the controller 4420 or a separate image processor (notshown).

An image frame processed by the camera 4480 may be stored in the memory4440 or externally transmitted through the communication module 4490.The device 100 may include at least two cameras 4480 according to anexemplary embodiment.

The communication module 4490 may include at least one component forcommunication between the device 100 and at least one external device2120. For example, the communication module 4490 may include at leastone of a short-range wireless communicator 4491, a mobile communicator4492, and a broadcasting receiver 4493. According to an exemplaryembodiment, components included in the communication module 4490 are notlimited thereto. For example, the communication module 4490 may includea component for performing at least one communication described abovewith reference to the network 2110 of FIG. 21.

The short-range wireless communicator 4491 may include a module capableof performing at least one communication from among Bluetoothcommunication, Bluetooth low energy (BLE) communication, near fieldcommunication (radio frequency identification (RFID), wireless localarea network (WLAN) communication (Wi-Fi), Zigbee communication, Ant+communication, Wi-Fi direction (WFD) communication, and ultra wideband(UWB) communication. A communication module included in the short-rangewireless communicator 4491 according to an exemplary embodiment is notlimited thereto. For example, the short-range wireless communicator 4491may include an infrared data association (IrDA) communication module.The short-range wireless communicator 4491 may use lower power Bluetoothcommunication when receiving a beacon signal.

The mobile communicator 4492 transmits and receives a wireless signal toand from at least one of a base station, an external device, and/or aserver on a mobile communication network. Here, the wireless signal mayinclude any one of various types of data according to transmission andreception of a voice call signal, an image call signal, or atext/multimedia message.

The broadcasting receiver 4493 receives a broadcasting signal and/orbroadcast-related information from an external source through abroadcasting channel. The broadcasting channel may include at least oneof a satellite channel, a terrestrial wave channel, and a radio channel,but is not limited thereto.

The communication module 4490 may transmit an avatar-based imagegenerated and/or changed by the device 100 according to an exemplaryembodiment to the external device 2120. The communication module 4490may receive at least one of user information, user modeling information,an avatar-based image, service information, coordinate information perobject, and combining condition information of objects from the externaldevice 2120, according to an exemplary embodiment. Information, data,and a signal transferred through the communication module 4490 is notlimited thereto.

FIG. 45 is a detailed functional block diagram of the controller 4320 or4420 according to an exemplary embodiment. For convenience ofdescription, FIG. 45 is described based on the controller 4420 of FIG.44, and description thereof may also be applied to the controller 4320of FIG. 43.

Referring to FIG. 45, the controller 4420 includes an informationcollecting processor 4501, a user modeling information generatingprocessor 4502, an image generating processor 4503, an image displayprocessor 4504, an image changing processor 4505, and a user modelinginformation changing processor 4506.

The information collecting processor 4501 may collect user information.The information collecting processor 4501 may collect the userinformation based on information received through the user input unit4411, the sensor module 4430, or the communication module 4490. The userinformation has been described above. A user input received through theuser input unit 4411 may be based on selecting an object in a form of animage provided by the information collecting processor 4501, but theuser input according to an exemplary embodiment is not limited thereto.The information collecting processor 4501 may store the collected userinformation in the memory 4440 or transmit the collected userinformation to the external device 2120 through the communication module4490.

The user modeling information generating processor 4502 generates usermodeling information by using the user information collected by theinformation collecting processor 4501. The user modeling information maybe configured as described above with reference to FIG. 3 or 24A. Theuser modeling information according to an exemplary embodiment is notlimited thereto. The user modeling information generating processor 4502may store the generated user modeling information in the memory 4440 ortransmit the generated user modeling information to the external device2120 through the communication module 4490.

The image generating processor 4503 generates an avatar-based imageaccording to an exemplary embodiment based on the user modelinginformation generated by the user modeling information generatingprocessor 4502. The avatar-based image is an image symbolizing a user.The image generating processor 4503 may store the generated avatar-basedimage in the memory 4440 or transmit the generated avatar-based image tothe external device 2120 through the communication module 4490. An imagestored in the memory 4440 or the external device 2120 may be temporarilystored therein.

The image display processor 4504 displays the avatar-based imagegenerated by the image generating processor 4503 through the display4412.

The image changing processor 4505 changes the avatar-based imagedisplayed through the display 4412, according to an exemplaryembodiment. After the avatar-based image is changed, the image changingprocessor 4505 may update the avatar-based image stored in the memory4440 or the external device 2120.

The user modeling information changing processor 4506 updates the userinformation stored in the memory 4440 or the external device 2120 basedon the avatar-based image changed by the image changing processor 4505.

The controllers 4320 and 4420 of FIGS. 43 and 44 may include greater orfewer components than those shown in FIG. 45. For example, when userinformation is received from the external device 2120, the controller4320 or 4420 may not include the information collecting processor 4501.

When user modeling information is received from the external device2120, the controller 4320 or 4420 may include a user modelinginformation receiving processor instead of the information collectingprocessor 4501 and the user modeling information generating processor4502, and a user modeling information transmitting processor instead ofthe user modeling information changing processor 4506. For example, whenan avatar-based image based on user information is received from theexternal device 2120, the controller 4320 or 4420 may include an imagereceiving processor instead of the information collecting processor4501, the user modeling information generating processor 4502, and theimage generating processor 4503.

As described above, the detailed functional block diagram of thecontroller 4320 or 4420 of FIG. 43 or 44 is not limited to that of FIG.45. Also, functions of the external device 2120 may be determined basedon the detailed functional block diagram of the controller 4320 or 4420.

Methods according to one or more exemplary embodiments may be realizedvia a computer program. For example, the methods may be executed by anapplication for easily changing, adding, and/or checking userinformation based on an image indicating the user information, theapplication installed in the device 100. The computer program may beoperated under an operation system environment provided in the device100. The device 100 may write the computer program on a storage medium,and read the computer program from the storage medium to use thecomputer program.

At least one program including commands for performing, by a computer, amethod according to an exemplary embodiment may be written ascomputer-readable codes on a non-transitory computer-readable recordingmedium. The non-transitory computer-readable recording medium is anydata storage device that can store data which can be thereafter read bya computer system. Examples of the computer-readable recording mediuminclude read-only memory (ROM), random-access memory (RAM), CD-ROMs,magnetic tapes, floppy disks, optical data storage devices, etc. Thecomputer-readable recording medium may also be distributed over networkcoupled computer systems so that the computer readable code is storedand executed in a distributed fashion.

It should be understood that exemplary embodiments described hereinshould be considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each exemplaryembodiment should typically be considered as available for other similarfeatures or aspects in other exemplary embodiments. While one or moreexemplary embodiments have been described with reference to the figures,it will be understood by those of ordinary skill in the art that variouschanges in form and details may be made therein without departing fromthe scope as defined by the following claims.

What is claimed is:
 1. A device for managing user information, thedevice comprising: a display; a user input module: and a processorconfigured to: obtain heath information of a user of the device, controlthe display to display an avatar-based image according to the obtainedheath information, receive a medical information request input via theuser input module; control the display to display medical informationrelated to the obtained heath information in response to receiving themedical information request input, receive a user input for selectingmedical information from among the displayed medical information via theuser input module, control the display to display information regardingthe selected medical information in response to receiving the userinput, and update the user information according to information receivedvia the user input module based on the displayed information.
 2. Thedevice of claim 1, further comprising a sensor configured to detect thehealth information of the user, and transmit the health information tothe processor.
 3. The device of claim 1, further comprising acommunicator configured to communicate an external device, wherein theprocessor obtains the heath information from information received viathe communicator.
 4. The device of claim 1, further comprising a memorystoring a health application, wherein the processor obtains the heathinformation by executing the health application.
 5. The device of claim1, wherein the information received via the user input module based onthe displayed information includes appointment information, and whereinthe updated user information includes user schedule information.
 6. Thedevice of claim 1, wherein the processor is further configured to:receive a user input indicating information to be added to theavatar-based image via the user input module, control the display todisplay an input window for receiving an input of the information to beadded, and add an image corresponding to the information receivedthrough the input window to the avatar-based image.
 7. A method ofmanaging user information, wherein the method is performed by a device,the method comprising: obtaining heath information of a user of thedevice; controlling a display of the device to display an avatar-basedimage according to the obtained heath information; receiving a medicalinformation request input via a user input module of the device;controlling the display to display medical information related to theobtained heath information in response to receiving the medicalinformation request input, receiving a user input for selecting medicalinformation from among the displayed medical information via the userinput module, controlling the display to display information regardingthe selected medical information in response to receiving the userinput, and updating the user information according to informationreceived via the user input module based on the displayed information.8. The method of claim 7, wherein the obtaining of the healthinformation comprises detecting the health information of the user byusing a sensor of the device.
 9. The method of claim 7, wherein theobtaining of the health information comprises obtaining the heathinformation from information received from an external device via acommunicator of the device.
 10. The method of claim 7, wherein theobtaining of the health information is performed by executing a healthapplication stored in a memory of the device.
 11. The method of claim 7,wherein the information received via the user input module based on thedisplayed information includes appointment information, and wherein theupdated user information includes user schedule information.
 12. Themethod of claim 1, further comprising: receiving a user input indicatinginformation to be added to the avatar-based image via the user inputmodule; controlling the display to display an input window for receivingan input of the information to be added; and adding an imagecorresponding to the information received through the input window tothe avatar-based image.
 13. A non-transitory computer-readable recordingmedium having recorded thereon at least one program which includescommands for executing a method of managing user information, whereinthe method is performed by a device, the method comprising: obtainingheath information of a user of the device; controlling a display of thedevice to display an avatar-based image according to the obtained heathinformation; receiving a medical information request input via a userinput module of the device; controlling the display to display medicalinformation related to the obtained heath information in response toreceiving the medical information request input, receiving a user inputfor selecting medical information from among the displayed medicalinformation via the user input module, controlling the display todisplay information regarding the selected medical information inresponse to receiving the user input, and updating the user informationaccording to information received via the user input module based on thedisplayed information.
 14. The non-transitory computer-readablerecording medium of claim 13, wherein the obtaining of the healthinformation comprises detecting the health information of the user byusing a sensor of the device.
 15. The non-transitory computer-readablerecording medium of claim 13, wherein the obtaining of the healthinformation comprises obtaining the heath information from informationreceived from an external device via a communicator of the device. 16.The non-transitory computer-readable recording medium of claim 13,wherein the obtaining of the health information is performed byexecuting a health application stored in a memory of the device.
 17. Thenon-transitory computer-readable recording medium of claim 13, whereinthe information received via the user input module based on thedisplayed information includes appointment information, and wherein theupdated user information includes user schedule information.
 18. Thenon-transitory computer-readable recording medium of claim 13, whereinthe method further comprises: receiving a user input indicatinginformation to be added to the avatar-based image via the user inputmodule; controlling the display to display an input window for receivingan input of the information to be added; and adding an imagecorresponding to the information received through the input window tothe avatar-based image.